PRE2020 3 Group1

Project O.W.L.

<div#bodyContent sup {

   font-size: smaller;
   vertical-align: baseline;
   position: relative;
   bottom: 0.33em;

}

1. bodyContent sub {

   font-size: smaller;
   vertical-align: baseline;
   position: relative;
   bottom: -0.25em;;

}>

Team Members

Name	Student number	Department
Tristan Deenen	1445782	Computer Science
Jos Garstman	1454722	Mechanical Engineering
Oana Radu	1325973	Computer Science
Ruben Stoffijn	1326910	Biomedical Engineering
Daniël van Roozendaal	1467611	Medical Science & Technology

Introduction

With dry summers becoming more and more frequent in the Netherlands, the risk of wildfires is becoming substantial. Last summer, the biggest ever national wildfire destroyed 800 hectare of Deurnese Peel. Eventually, the fire department got the fire under control.

Fire in the Deurnese Peel from above

The Dutch fire brigade is currently experimenting with drones to fight fires. With a number of drone teams already operational, the fire department is making big strides in innovation. Currently, drones are used to support teams while fighting fires or evacuating buildings. The fire department has indicated that flying drones for fire prevention would save the most resources. They are not currently doing this, mainly because of a responsibility gap involving staatsbosbeheer and the fire department (see interview). Since looking after national parks is the responsibility of staatsbosbeheer and the fire department are currently experimenting with drones, we propose a plan for cooperation: Overhead Wildfire Locator, or O.W.L. for short, where drones can be deployed to survey national parks during extreme drought and fire hazard. We will go over ways to automatically detect starting fires. A simulation will be performed to estimate drone response time and coverage. The drone specifications and hardware limitations will be explored. The level of feedback and control that the user will have is suggested. The legal and ethical questions regarding semi-automatic drone flight above nature reserves are addressed. We try to keep everything as general as possible, but to answer more specific questions we use the Loonse and Drunense duinen as a case study. All the points of attention come together to form a propositional protocol, that staatsbosbeheer can realistically use in the summer of 2021 during extreme weather. Possible future developments are discussed in a roadmap.

Forest fires and detection

With wildfires being such an increasing problem, we should take a look at the cause, the way it spreads and ways of detecting it (also Taskforce Natuurbranden), before we can give a consult on how to prevent and combat it.

It is believed that over 95% of all wildfires are not caused naturally but instead caused by humans, intentional or by accident. The main reasons of ignitions found in a research in Santa Monica and San Diego are arson, power lines and sparks by cars or equipment.

The average windspeed in the Loonse en Drunense Duinen over the year is around 4.0-4.5 meters per second (14-16 km/h) . Loonse and Drunense Duinen is a combination of forests, grasslands and sandbanks so the speed of spreading of the fire will vary per location.

A rule of thumb used by the fire department in the Netherlands states that the speeds of the fire equals the speed of the wind (in meters per second) times 100 with the result in meter per hour. In open terrain (heather, grass) the speed is doubled . Also mentioned in this source: fire spreads with an angle of about 48 degrees in the direction of the wind.

Using this rule of thumb, we would get a speed of up to 450 meters per hour in the forest and 900 meters per hour in the grasslands.

Area	Wind speed	Fire speed
Forest	4.5 m/s	0.45 km/h
Grass/heather	4.5 m/s	0.9 km/h
Sand	4.5 m/s	0 km/h

The temperature when a material will burst into flames (also called the flash point) of wood is 572 degrees Fahrenheit (300 degrees Celsius) . Given the difference between this temperature and the average maximum surrounding temperature in the Loonse and Drunense Duinen being 23.5-24.0 degrees Celsius, any fire will easily be detected by an infrared sensor. Even in extreme temperatures of 30 degrees Celsius or higher, the difference with flaming wood will be around 270 degrees Celsius. This difference being so high means that even small, starting fires can be detected.

Forest fire detection is typically performed by some combination of watchtowers, aerial detection patrols and satellite imagery. Watchtowers have to be carefully placed to be have enough visibility to be useful, they are often expensive to build and it is not very flexible as the view is permanently decided once a watchtower stands. Satellite images play an important role if fire management and strategic fire intelligence as it covers a lot of territory and can detect medium to large fires using (semi) automated algorithms. Some exceptional satellites can even be used to detect small fires but overall satellite visits are too infrequent and the image resolution makes detection of small, developing fires difficult. The time lag of a user receiving data from satellites can take hours and weather can limit availability of satellite data, making it inconsistent and unusable for fire detection.

Heat detection is an obvious method of detecting fires in natural areas. Normal temperatures are on the thermal infrared (TIR) region. Higher temperatures like that of typical fires, are on the mid-wave infrared (MWIR) of shorter wavelengths. Many sensors detect near infrared (NIR) and shortwave infrared (SWIR) and can be used for detecting fires. Large clouds of smoke, caused by forest fires, can be detected on standard cameras when there is enough light but these still have to be identified by humans. In the dark, fires can also be detected using night vision goggles (NVG). Thermal imaging seems to be the optimal way of detection compared to the alternative. More about the thermal camera will be explained later on.

Detection speed impact

To illustrate the importance of a quick response time, we will show the destructiveness of exponential growth in a hypothetical fire spreading situation. If we divide the response time up in periods of 10 minutes, we can calculate how much effect each period has on the size of a wildfire. For this calculation, we are going to approximate a fire as an expanding circle. Every time 10 minutes pass, the amount the fire grows increases, because a later increase in diameter will cover more area than the last. If we use the speed of fire spreading being 0,9 km/h which gives us around 0,15 km for every 10 minutes. The first time period starts as a single ignition point. After this first time period we have a circle with radius 0,15. The radius increases with the same amount every period over the first hour and a half. The results are shown in the table below.

Time period	Radius (km)	Area (km^2)	Area difference (km^2)
1	0,15	0,07	0,07
2	0,3	0.28	0,21
3	0,45	0,64	0,36
4	0,6	1,13	0,49
5	0,75	1,77	0,64
6	0,9	2,54	0,73
7	1,05	3,46	0,92
8	1,2	4,52	1,06
9	1,35	5,73	1,21

Every 10 minutes that passes are more destructive than the last. The differences may seem small, but that is the way with small, starting fires. The difference every 10 minutes makes is clear to see in this table and that is what is most important. During the first time period the fire only grows 0,07 square kilometres and in the last time period the difference is around 1,2 square kilometres. Here we see how much of a difference early detection of wildfires can make.

Taskforce Natuurbranden

As a result of the escalated fires at the Kalmthoutse en Strabrechtse Heide, a taskforce was formed called “Taskforce Natuurbranden” (Taskforce Wildfires) . This taskforce was initiated by Bas Dikmans in North-Brabant. It is specialized in analysing what has happened before and how it can be prevented in the future. The local municipality, fire department and the area administrator have to perform what is necessary in the area themselves, the taskforce is only their collaboration when they are involved in planning. Once a year, mayors have to pledge accountability to the King’s Commissioner.

In 2020 multiple wildfires collectively burned over 1000 acres of wild areas in Brabant. The taskforce used these fires to conclude that the long drought, heat and strong winds form a risk in the Netherlands and that wildfire prevention is a form of climate adaption. Important points of wildfire prevention include local fire brigade knowing their way in the area, emergency services easily being able to go in and having enough water for extinguishing close by. Not only easy access for controlling fire is important. The kind of trees in the area can be a factor for example. Conifers are more flammable than deciduous trees. Since human ignition is the main cause of wildfires, more human visits means a bigger risk of fires but at the same time this means faster notifications to the fire department. Mobile service is very important in these risky areas to get the notification through.

Keeping nature wet and placing so called water buffering areas is very important for prevention. According to Robby Brekelmans the high sandbanks in North-Brabant are very susceptible to drought, that’s why it’s so important to shape the water system in a way that the water can be held in place to avoid this drought.

The taskforce has made risk assesments for some specific areas. If an area is deemed to have a greater risk for wildfires, this could be taken into account before scouting for wildfires. The more information is available for a specific area, the better wildfires can be detected by planning specific routes for the drone.

Type of drones

Currently there are around four main types of drones. Namely single-rotor, multi-rotor, fixed-wing and fixed-wing hybrid drones. All these types have their own strengths and weaknesses and depending on the usage one might prefer one over the other. In this section a run-down of these four main types will be given, mentioning both their pros and cons.

Multi-rotor drones

DJI matrice 600 pro

Multi-rotor drones are surely the most popular drones on the market. They are good starter drones because they are cheap and easy to use. They have multiple rotors keeping them up, can take-off vertically and have great control in the sky with the option to hover in place, this way the surrounding area can be observed for a longer time.

There are, however, also quite some downsides to this type of drone. A multi-rotor drone is constantly fighting gravity to stay up in the air. This requires a large amount of energy and is generally not very efficient. A multi-rotor drone with a light payload can only stay in the air for around 20 to 30 minutes. On top of this the design of this type of drone also limits their speed. Currently they are also restricted to the use of electric motors.

All in all a multi-rotor drone is a cheap way to get an aerial view of the nearby area for a short amount of time, but they are not suited for long duration flights over long distances

Single-rotor drones
Single-rotor drones are of course quite popular in manned aviation, but are not that widely available in the drone world. It has a single rotor with a tail rotor to control direction. A single-rotor drone is much more efficient than a multi-rotor drone because it has one much larger rotor instead of several smaller ones. Additionally they can be powered by a gas motor for even longer flight times and they can carry much heavier payloads than multi-rotor drones.

The downsides of single-rotor drones are that they are much more complex and harder to handle. The cost and maintenance are also quite high because of the mechanical complexity of the copter. Finally the longer, sharper blades of the single-rotor are much more dangerous and can deal some serious damage.

Fixed-wing drones

C-Astral Bramor SAR

The main difference between rotor-drones and fixed-wing drones is that a fixed-wing drone uses wings to generate lift instead of rotors. This way a drone only needs to create forward momentum and takes advantage of physics to stay up in the air and increase efficiency and speed. because of this big increase in efficiency and speed fixed-wing drones can stay in the air for longer, up to one hour, and thus cover much larger distances than rotor drones. Additionally a gas engine can be used as power source for even better endurance. This way some fixed-wing drones can stay in the air for more than 12 hours.

The main disadvantage of fixed-wing drones is that they cannot hover on the spot and are always moving forward. It is much more difficult to survey one area and take a closer look if more info is needed. Also take-off and landing is more difficult because of the high speeds. Depending on their size a runway or catapult are needed to launch them into the air and either a runway, parachute or net is needed to retrieve them again. Some smaller drones are capable to be launched by hand and land safely on their belly in a field. Some other cons are their higher costs and difficulty to control.

ALTI Endurance

Fixed-wing hybrid drone
A fixed-wing hybrid drone combines the main advantages of rotor and fixed-wing drones. These types of drones are also called VTOL's, which stands for Vertical Take-Off or Landing. These types of drone do not need a runway or catapult to take-off but can tilt their rotors or switch between rotors to take-off and land vertically, but fly like a fixed-wing drone. These types of drones are relatively new and upcoming, which makes them more costly.

The combination of the two types of drones means it is not perfect in either hovering or forward flight, but the option to hover greatly increases its functionality.

Drone Functionalities

For the drones to be able to successfully survey an area and detect hotspots they will need to have certain qualities. In this chapter a list of functionalities will be given and explained in further detail. Several options will be laid side to side and recommendations for the drone will be given. The following functionalities are necessary for the project O.W.L. drone:

1. Detection of temperature difference to spot beginning fires
2. Detection of surrounding environment for additional information
3. Path-finding and collision detection
4. Communication of data with main operating base/main operator
5. Large range and long battery life to patrol surrounding area
6. High visibility for surrounding environment
7. Low noise for surrounding environment

Detection of temperature difference to spot beginning fires

Forest fire from the perspective of a drone

To prevent a full scale wildfire from occurring, hotspots have to be detected as fast as possible. The detection of hotspots can be done by measuring the temperature of the environment, since a beginning fire will have a higher temperature than its surroundings. This temperature difference can be measured in a multitude of ways and sensors, but not all are fit to be attached to a drone. For a sensor to be effective on a drone it needs to be long range, accurate and to be relatively stable at higher velocities.

For the case of detecting areas with increased temperatures from a drone, only one method seems viable, a thermal camera. An infrared or thermal camera is a non-contact sensor which measures the radiation of objects. When attached to a drone it can detect this radiation from a long range with fairly high accuracy from multiple objects. Then in one image the different temperatures in the area can be seen. In the past it has been used in tasks such as surveillance, search & rescue and also firefighting. Other methods such as classic thermometer could be used to measure temperature differences, but this would lead to some difficulties. A drone will have to get quite close to its target for a thermometer to detect a temperature difference. Furthermore, it can only measure one temperature at a time. There are many different specifications for a thermal camera and dependent on the use of the camera the choices made will be different. An overview of the specifications and which are important for locating hotspots will be discussed in the following paragraph.

Thermal camera

By patrolling and scanning the area, a thermal camera can create and compare thermal images to detect hotter areas in the landscape. Closer inspection can be done to confirm if a starting fire is present. There are some key aspects to think about when deciding which thermal camera is a good candidate. The main specifications are the following:

1. Weight
2. Radiometricy
3. Degrees of freedom
4. Thermal range and sensitivity
5. Infra-red resolution
6. Spectral range
7. Camera lens

First of all its weight. A bigger, heavier camera needs a bigger drone to carry it. At the same time a heavier payload will cause for a shorter air time of the drone and thus the need for a bigger battery pack. When deciding upon which camera to choose, a lighter camera would be optimal.

Secondly radiometric vs non-radiometric. A non-radiometric thermal camera only display a thermal image while a radiometric camera also provides the temperature measurements. For the purpose of finding hotspots and fire-prone areas a radiometric camera greatly improves performance. These cameras can give a warning whenever a certain threshold temperature is crossed, at which point the drone could act immediately.

Next is a stable camera or one attached to a gimbal. When a camera is attached to a gimbal it gives the drone or operator more control over the cameras movement. A gimbal makes it easier to survey the area without having to turn the entire drone

Flir Vue Pro R thermal camera

Furthermore the range and thermal sensitivity of the camera. The range means the entire span of temperatures the camera can measure. For the purpose of finding fire-prone areas a lower range is sufficient. The thermal sensitivity of the camera describes the smallest temperature difference you can see with the camera. This means that the lower the number the better the thermal sensitivity is and the better it can detect temperature differences. For this case a low sensitivity is not necessary, since there will generally be wide temperature differences.

IR resolution is also important when choosing a camera. A higher resolution means that the image created contains more detail and information. Especially for longer distances, like a drone surveying the area, a higher resolution is necessary to measure everything in enough detail.

Finally the spectral range of the camera. This is the range of wavelengths that the camera detects, measured in micrometres. Almost all thermal cameras are longwave cameras which have a spectral range of 8 to 14 micrometres and are appropriate for finding fires.

Some common and representative thermal camera manufacturers for UAV's are FLIR, WIRIS security and Yuneec.

Camera lens

Another important part of the thermal camera is the type of lens. The lens has an impact on the resolution and field-of-view of the camera. The type of lens that is used depends heavily on the situation it will be used in. In the case of a fire-seeking drone, the camera will be at a reasonably high altitude and will need to scout as much area in as little time as possible for maximum efficiency, all the while the detail has to be high enough to detect temperature changes.

First of all we have the height of the drone. Since the distance to the ground, the target, will be relatively large the thermal images might not that have that much detail. To increase this detail the thermal camera will need a higher resolution. A higher resolution means that there will be more pixels in the image, therefore the area that one pixel represents becomes smaller and thus there is more detail in the image.

Next the field-of-view of the camera(FoV). FoV means the amount of degrees the camera can observe. A bigger FoV therefore means that the camera can observe more at once, however smaller objects at a larger distance will be more difficult to spot. For the UAV to be able to patrol the largest area a wide FoV is optimal.

To determine how accurate the thermal camera is at a distance, we need to calculate the spot-size ratio(SSR). The SSR uses the instantaneous field of view(iFOV) and the distance to the target to determine how far the camera can be from its target and still get an accurate measurement. The iFOV an angular projection of just a single pixel of the thermal camera and to calculate the iFOV the following formula is used:

iFOV = (FOV/#_of_pixels)*[(3.14/180)*1000]

Here the number of pixels should coincide with the direction of the FOV. Using the highest resolution of 640x512 and a FOV of 45x37 degrees will result in a iFOV of 1.227 mRad. Converting this to meters we get an SSR of 0.012:10, meaning that an area of 1.2 by 1.2cm can be measured from 10 meters away. However this is the theoretical SSR and often not an accurate representation of reality since it is a single pixel measurement. Such a measurement may be inaccurate due to:

• The development of bad pixels
• Solar reflections causing false positives
• Distortion in the optics

Therefore instead of a 1x1 pixel area a 3x3 pixel area is recommended to give a more accurate representation. The iFOV can simply be multiplied by three giving an SSR of 0.037:10m. This value scales linearly and the following values can be determined:

SSR of 640x512 and 45^ox37^o

Target size (cm)	Distance to target (m)
3.68	10
18.40	50
36.79	100
44.16	120
55.18	150
91.97	250
100	272

When deciding upon which type of camera and lens to use a balance between the amount of detail and the FoV has to be found. For the finding of hotspots the amount of detail is relevant, but the image does not have to be extremely detailed. Once a general hotspot has been detected the drone can be used to investigate further in that area. Still the highest possible resolution is recommended for maximum detail possible. A standard FoV of 45 degrees is recommended to be able to scan a wide area while still keeping high detail. With these specifications we can see that flying at a height of 272 meters a target of 1 meter can still can be measured.

Detection of surrounding environment for additional information

On top of a thermal camera sensing the temperatures below the drone, an additional camera is used to for observation of the surrounding area. This camera can be used by an operator for further inspection or to fly the drone if necessary. For example, once an increased temperature is detected, a drone operator could check if there is no false positive at play, by taking manual control using the camera to pilot the drone and check the area. Furthermore, the camera could check the horizon for smoke plumes, indicating a fire could be present at that location.

Path-finding and collision detection

For autonomous patrolling of a certain area, sufficient GPS and navigation systems have to be in place. These systems should be able to follow specific flight paths which are predetermined or autonomous flight paths could be implemented based on previous data of hotspots. Collision detection should also be part of the UAV's sensors to avoid collision with wildlife, other aerial vehicles or other objects. Most consumer drones have integrated collision detection, however typically only in front of the drone. There are only a few drones that have collision detection in all six directions.

Communication of data with main operating base/main operator

Once a hotspot or starting fire has been detected, this information has to be relayed to the main operating base or fire brigade. This communication should be fast, high quality and over a large range. Communication with a drone is usually done via radio-frequency transmissions. These kind of transmissions can include the location, battery life, distance, payload information and much more data, including (live) video. Next to data transmission, the UAV is also controlled via radio-frequencies. Three main data link systems can be distinguished for drones:

1. UAV control
2. Video transmission
3. Telemetry

Radio communication

Data communication with drones

Before diving deeper into the data link systems, a further look into radio frequencies is needed. There are a couple of common frequencies with each their own speciality based on their capabilities regarding object penetration and range. In general, the lower the frequency, the longer the wavelength and the greater the penetration into objects and the longer the range. However, a lower frequency means there is limited maximum data rate, meaning only lower quality data can be transmitted. The most common radio frequencies used are:

1. 900MHz
2. 1.3GHz
3. 2.4Ghz
4. 5.8Ghz

For UAV control the most popular frequencies are 2.4Ghz and 900Mhz. Most consumer drones fly on 2.4 Ghz, which has a range of around 1.6 to 6.4 kilometers. However, the penetration of this frequency is quite low and it will have difficulty flying in areas with many buildings, mountains or forests. 900Mhz has a much longer range from 20 to sometimes even 100 kilometers, including a much higher penetration.

Video transmission is a bit more tricky, since 900MHz is unable to transmit good quality video. The most popular frequencies used here are 1.3GHz, 2.4GHz and 5.8GHz. The same properties from before hold, 1.3GHz has a long range up to 60 kilometers and high penetration, but will only be able to transmit poor video quality due to the limited data rate. 5.8Ghz is the opposite, it will give the clearest and highest video quality, however it's range will be around 10 kilometers with low penetration capability. 2.4GHz is the balanced option, giving decent video quality on a range of around 25 kilometers. The downside of this frequency is that it is often used for controlling the drone, as discussed previously. Only one can use this frequency or there will be interference.

For remaining data lower frequencies such as 400MHz and 900MHz are used. This data does not need the high data rate of video so lower frequencies can be used for maximum range and penetration.

LTE Communication

LTE communication is still an experimental way of communication for drones and currently not yet fully operational. LTE stands for Long-Term Evolution and is part of the cellular network. The idea is to control and communicate with UAV's via 4G as an alternative to radio control. The advantages of LTE is that the network is connected to satellites and broadcasted via towers. This greatly improves the distance limitations and improves long-range communication with drones. Currently, the challenges for this type of communication are that these networks can be quite unstable and good network connection is needed for real-time drone control. On top of that, to be able to use this type communication, some additional infrastructure is needed. There are however promising experiments and research papers on the use of LTE communication for drones and it will likely be a viable option in the near future.

Large range and long flight time to patrol surrounding area

To be able to monitor an entire nature reserve or forest, a big enough range to cover the area is needed. The range of a drone not only depends on the speed and flight time of the drone, but also on its data communication range. First of all, the speed and flight time of the drone will be discussed. In the chapter Type of drones a distinction between flight range has already been made. Fixed-wing drones use their wings to create lift and therefore preserve flight time. Additionally, flight time is dependent on multiple other factors such as drone weight, wind, altitude and camera use.

Factors such as wind are of course uncontrollable, but to other factors, such as weight, extra attention should be paid. The weight of the drone should be kept as low as possible to increase flight time. A lighter drone means the propellers or wings need to create less lift to stay up and the drone can generate more speed. Camera use is also an important factor for the case of hotspot detection, since it will require continuous camera use. This will take a toll on the battery life.

Depending on the type of drone, the flight time will be between 20 to 30 minutes for rotor-designs up till 1 hour for electric fixed-wing drones. After this time limit, the drone will have to return to recharge or change its battery. During this time there will be a gap in the surveillance of the area. To get continuous or near continuous monitoring multiple drones or pre-charged battery packs should be available. The moment a drone returns to recharge another drone can leave to take over its job. Another possibility would be for an operator to quickly switch battery packs so it can continue on its way. Some research is being done into automating the process of swapping battery packs for drones. In the future this could be implemented to reduce workload on on-site operators or completely automate the operation.

Another option would be to use a combustion engine powered drone, which prolongs the flight time to up to 12 hours. Refilling will be less frequent but requires more work. This could likely also be automated. The price of maintaining such a drone will increase do to the higher complexity of the machinery and the cost of the fuel. Other downsides of combustion engines are the increased noise and the emission of greenhouse gasses.

The flight time is often not the limiting factor of drone range, but the data communication range is, as discussed in the previous section. Flight time does determine the area that can be covered within the range of the drone.

Range does not only depend on how big the area you need to cover is, but also from where you are controlling the drone. If the main base is in the middle of the area, the radius does not have to be as large as when the main base is on the outskirts of the area. However, when surveying a nature reserve, a drone station in the middle of a park could not be a possibility because of disturbing wildlife or building restrictions.

High visibility for surrounding environment

The UAV should have high visibility so wildlife or other aerial vehicles can see it coming. This can be done by means of lights and reflective strips. Also, bright eye-catching colours can be used for the drone to stand out from the environment. To preserve battery life, the use of lighting should be kept to a minimum and preference is laid upon bright colours and reflective materials.

Low noise for surrounding environment

Noise should be kept to a minimum if the UAV is flying in nature, not only for surrounding wildlife but also hikers or other civilians. Current consumer drones generally create noise between 70 and 80 decibels. This equal to quite loud traffic and excessive noise levels generally start at around 85 dB. The level of noise depends on distance and the dB level will drop over distance. The rule-of-thumb for the drop-off distance is that sound decreases by 6 dB every time the distance doubles.

Noise drop-off rate

Now how a quiet a drone has to be depends on the ambient noise levels which differ per area and time of the day. Generally, during day time in a busy city center, the ambient noise level will be much higher than at night in a rural forest. Since the surveillance drone will be flying in nature reserves, the noise it generates should be quite low. The ambient noise level in a rural area generally does not drop below 45 dBA at day and 35 dBA at night. The ideal noise range for a drone would therefore be 35-45 dBA.

To reach this level, the drone could of course fly very high, but this would decrease the detail of the thermal camera and would make it difficult to detect hotspots. Therefore, the drone has to be designed in such a way that the noise level is reduced. Reduction of propeller noise can be achieved by increasing the diameter of the propeller while also reducing the rotational speed. This means that the blade tip speed is reduced while maintaining enough thrust.

Finally, there is a difference between electrically powered and combustion engine powered drones. Generally, the electric UAVs are much more silent than the combustion engines. To minimize noise pollution, an electrically powered drone is recommended.

User requirements

Stakeholders

Scouting for fires preemptively benefits multiple stakeholders, and their responsibilities are intertwined. Staatsbosbeheer’s primary task is maintaining nature parks across the Netherlands, so they are responsible for monitoring their reserves closely for fires. The people who hold this task are the foresters, while the Dutch fire department is responsible for extinguishing fires. During an interview they have stated that the largest strides can be made in fire prevention. Since wildfires require great resources, preventing them will be beneficial.

Users

In the early stages of transitioning to drone monitoring for nature reserves, the users will have to be certified drone pilots. Teams of which the fire department already deploy nationwide. This is necessary because of strict European drone regulations. When, in the course of the coming years, more reliable autonomous drones can be deployed, the user requirements will most likely become less strict. Allowing users who are less qualified to use the system. Because of this predicted shift in userbase, user requirements will change and evolve over time.

Degrees of user requirements

Because drone technology and legal restrictions change over time, drone usage is dived into multiple stages. Note that these stages are not strictly exclusive and overlapping is possible.

Total control

This stage is an example of how drones work in the field currently, for example at the fire department. Current drone regulations require certified pilots. They are this stage's users. Combatting fires usually requires a team of two or three professionals. One for piloting the drone, another for camera controls. The last optional member handles technical problems and helps with the drone setup. Since our goal is surveillance instead of reconnaissance, this solution requires the full attention of at least two pilots, for longer stretches of time. It goes without saying that this is very strenuous and monotonous work.

The feedback these pilots will recieve is very rudimentory. The drone controlling pilot gets access to images obtained from a front facing camera. This helps pilot the drone from beyond visual line of sight, as well as noticing smoke from far away. The second pilot has control of a thermal camera. They can rotate it and angle it up and down. They arguably have the hardest task, which requires constant evaluation of thermal images for dangerous temperatures. Both video signals are usually fed back to the controllers, but feedback to a computer system is also possible. GPS coordinates are also required, namely to more specifically pin down the location of wildfires and keep track of the drone's location along a flight path.

Semi-autonomous

This stage, semi-automous, aims to scale the drone team down to only two pilots. This is achieved by automating some simple procedures.

Drones that move automatically already exist, but are not used in any professional fire combatting applications. Since our goal is surveilance, drone movement is not complex and requires only large-scale routing. What we propose is that the pilot brings the drone to a height that oversees the entire nature reserve. Now, the pilot can activate automatic routing mode. This entails that the drone stays at a fixed height and flies directly to specific coordinates. These coordinates, or waypoints, can be manually selected by the pilot. This requires licensed sofware or custom software specifically made for this purpose.

The workload of the pilot will consist of the following tasks:

• Patrol route selection
• Manual drone takeoff
• Thermal camera control and image surveillance
• Manual drone landing

Feedback will be provided in two major ways: live camera feed from both cameras and real time GPS drone location on a map of the area.

Advanced semi-autonomous

This stage, as the name implies, is an advancement on the last. It eliminates more workload on the surveillance functionality. Since the camera motion is simple and easily programmable, it can be automated with an efficient search strategy in mind. It can sweep around by itself and scan a large area without the need for manual control.

The next step in automating the surveillance task is automatic fire detection, which is explained in detail in Forest fires and detection. The main idea is that the drone supervisor is alarmed when dangerous temperatures are detected. These cameras exist already commercially, mainly for use in factories and other work environments with possible failure due to high temperatures.

With current regulations, the person supervising the drone still needs to be a licensed pilot. Their only task requiring drone piloting is take off, landing, and taking over manual control when routing goes wrong. This greatly reduces the workload on the pilot's part.

Fully autonomous

A fully autonomous system would reduce the responsibility of the user to the bare minimum. The fullest extent of autonomy regarding drone surveillance reaches to complete unsupervised take-off and landing and guaranteed safeguards for GPS malfunction. Because the supervisor would not need to intervene, they do not need to be licensed pilot. This does, however, mean that the person activating the drone cannot be responsible for whatever actions it takes. You could agree that activating the drone in and of itself makes that person responsible, whether they have a drone pilot's license or not. Another party that could be responsible would be the manufacturer. If implementation of fully autonomous drones is only allowed with a complete guarantee for safety from the manufacturer, then they would be held responsible for any violations. Most realistically, both parties come to an agreement before activating the system. The manufacturer can provide guidelines within which the drone is guaranteed to behave safely. If the drone malfunctions while these guidelines are being honored, the manufacturer is responsible. If the guidelines are breached, the user is.

European drone regulations

Since December 31 2020, the Netherlands follow European drone regulations. These new regulations divide drones in 3 separate categories: Open/zero, specific and certified. Normal consumer or hobby drones usually fall in the open category. These drones have a few restrictions  :

• Maximum weight: 25 kg (at takeoff)
• Maximum height: 120 meters
• No transporting hazardous material
• No dropping materials
• Always have visual line of sight

There are subclasses for the first category, depending on the weight of the drone. Most relevant is subcategory A3 which concerns drones from 2kg – 25kg. With normal regulations this category cannot fly 150 meters near any living, trade, industry or recreational zones.

The next category, specific concerns flights that:

• May be near people
• May fly near airports
• May have a weight above 25kg
• May fly in inhabited environment
• May fly above a height of 120 meters
• May drop materials
• May fly beyond visual line of sight (BVLOS)

Drones deployed by the Dutch fire brigade fall under the specific category, and more than likely, will be used by the forestry department as well. Obtaining authorization for flight needs to be done at the national aviation authority. The Dutch fire department has broader permissions regarding drones because of the fact that they have a specific flight originization. Prior to this, the Dutch fire brigade had unique exemption from specific drone laws granted by the government. Before testing and small scale rollout of the surveillance procedure is takes place, such a temporary exemption can realistically be attained.

It is also worth noting that the people controlling these drones need to be certified pilots. The Dutch fire brigade already has an official training program for aspiring drone pilots . It would not be unrealistic to expand this training to other applications as well.

Captured data

Once images have been taken, there can be conflict. This depends on the fact whether or not it contains personal data. This is the case when someone is visible on the images and is recognizable with reasonable effort. With a thermal camera this would hardly be the case. Also, based on the distance between people on the ground and the angle (mostly seeing top down), recognizing people with the thermal camera would be very unlikely.

With a normal camera, however, it is quite possible that images are taken and people will be recognizable. In this case, there is personal data being recorded when someone is recognizable on the drone footage. This is an infringement on the privacy every person in entitled to.

Personal data can be proceeded to be used only if this data is used for the original goal of the footage. For example, when footage of a concert is taken for promotional purposes, it is allowed to use to use the footage, despite the fact that personal data is used. However, this footage is not allowed to be used later on for purposes like identifying people visiting the concert. This is a goal unrelated to the original goal of the footage.

Simulation

Part of Loonse en Drunense Duinen used for the simulation. The area is 1800x1200 squared meters.

To see how effective this addition to the Staatsbosbeheer is in helping them preventing wildfires, we have built a simulation to test exactly that. Furthermore, we can gain new insights on how effective new functionalites or ideas are.
The purpose of the simulation is to check whether a drone can detect small fires outside of cities and notify the forest rangers and/or the firefighters. We decided to make the simulation in NetLogo. One could also choose to program their own simulation environment. However, with NetLogo, we have instant acces to essential features, such as the turtles and patches, which allows us to directly create a drone unit and a programable landscape. Furthermore, we gain a fully functional user interface, and especially the function “import-pcolors”, which copies a picture on top of the patches and gives each patch the corresponding colour from the picture. Hence, NetLogo is perfect for this simulation.

We discuss the results of the simulation in Effectiveness of the current system. This is the file: File:Drone.zip

The environment

The simulation after the setup. The transparant white circles around the drone represent the vision of the two cameras.

The environment of the simulation consists of 2 turtles and 740000 patches. There are two breeds of turtles: drones (for the drone) and stations (for the base). Ideally, the base station is placed in the middle of the search area, to optimize the search. The drone has a “drones-own” variable called energy that keeps track of the energy of the drone. To copy the image of the satellite map, the function “import-pcolors” was used. The drone leaves from the base and scouts the area. When it leaves from the base, the drone has charged batteries. The setup of the simulation can be seen in the figure on the right.

All patches have 2 variables: grow and fire. Grow denotes whether the fire should be increased or not, while fire represents the intensity of the fire. In the beginning of the simulation, both variable are 0 for all patches. Grow takes values between 0 and 11, 11 symbolizing that the fire has to grow. For all the other values nothing happens. The variable fire take values from 0 to 5; 0 denotes there is no fire in the respective patch and the values from 1 to 5 denote the intensity of the fire, with 5 meaning the fire reached maximum intensity.

Realistic values

First, we chose the height at which the drone flies. As described in Camera lens, at a height of 272 m, the drone has a target size of 100 cm. For this simulation, the drone flies at a height of 250 m, so that the drone is definitely able to differentiate an area of 1 squared meter. Looking at the area of each patch, the target size is now one third of the patch area. Therefore, the drone can reliably check each patch for a fire. As the drone flies at such a high height, and after experimenting with different values, we have chosen to let the drone fly at a pace of 20 km/h or 5.6 m/s. At this flying speed, the drone flies slow enough to still give the drone enough time to process each patch well enough, while also being fast enough to be effective. Due to the possibility of excessive noise and random unfortunate events, we have opted to go for a lower flying speed, so that the results represent real-life scenario's in a more reliable way.

To keep track of time, we use ticks, instead of a continuous flow of time. This makes it easier to make sure that everything moves, grows or depletes at the right pace. To make the drone move the distance of one patch per tick, each tick represents 0.32 s. We have set the battery life to 1 hour. Again, we opt for a somewhat lower battery lifetime to make the results of the simulation more reliable. Now, dividing the battery lifetime by the time of each tick, we get that the battery lasts for 10800 ticks.

Concerning the speed of the fire, we have used the statistics as described in Forest fires and detection, so we used a fire speed of 0.45 km/h or 0.13 m/s. Of course, before the fire can be ignited, the temperature of the fuel for the fire must sufficiently high. This means that the drone could detect such a temperature anomaly, even though there is currently no fire. We simulated this by making the first patch of the fire take significantly longer to expand than every other patch. Lastly, we have set the maximum size the fire may have to 300 patches. If the fire gets bigger than this, the simulation run has failed. We have chosen for this number, because at around this point, the fire is likely to have been seen by other people, which should be the purpose of the drone.

Setting the fire and wind

The start of a fire is simulated by changing the colour of one of the patches to red. Then, its value of fire becomes -10, because it takes quite a lot for the fire to actually start, but the temperature will be high, as mentioned above as well. The coordinates of the fire are chosen randomly, so the fire can be placed anywhere in the chosen zone. To simulate the growth of the fire, both patches’ variables are used. The algorithm used to make the fire grow works as follows:

The fire can expand only to the patches next to the patch that is ‘on fire’, so the read patch. Hence, the following steps happen only for the red patches. First, the variable grow is compared to 11. While grow <= 11, grow gets increased by 1 and nothing else happens. When grow = 11, the fire intensity grow, so fire gets increased by 1 and grow gets reset back to 0. The variable fire has maximum value 5, so when fire = 5 it does not get increased anymore. Second, the variable fire is compared to 5. When fire = 5 it means the fire spreads according to the wind, which will be explained in the section below, and the colour of the corresponding patches gets changed to red and fire = 1 for those new patches.

To make the simulation more realistic, we introduced a variable wind , that represents the direction the wind flows. Wind can take values from 0 to 4; 0 = no wind, 1 = wind from west, 2 = wind from east, 3 = wind from north and 4 = wind from south. At the beginning of the simulation, wind gets a random variable from those mentioned above. The shape of the fire in those 5 cases can be seen in the picture below.

The shape of the fire depending on the direction of the wind.

The cameras

The drone is equipped with two cameras; one long-range thermal camera, and one short-range camera. For the user's convenience, the areas that the drone can scan are highlighted on the map. In the simulation we simplify the active field of view of the cameras by giving both cameras always a 360 degree field of view. As the drone moves relatively slow and flies high in the sky, the cameras have enough time to make a full rotation, before it has covered too much new ground.

The long-range camera thermal camera is responsible for spotting weird temperature anomalies that may cause or are already causing fires. This is simulated by making the long-range camera only scan for red patches. Note that the thermal camera cannot detect anything else on the ground, as it cannot see any details. This camera looks at an angle of at much 45 degrees upwards. Hence, its maximum range that the camera can see, is exactly the same as the height at which the drone flies, so 250 m.

The short-range camera is a normal camera. The functionality that this camera provides, is that, unlike the thermal camera, it can detect details about the ground and the forest. Hence, the drone can roughly check over which kind of biome it is flying. To make sure that these details do not get lost, the range of the camera is severely shorter than the previous camera.

Scouting

The route of the drone while scouting.

The drone starts from the base and heads to the north. Since the drone has the long range camera which spots anomalies, the drone does not have to visit all patches, but all patches need to be covered by the camera. Therefore, when the long range camera covered the topmost patches, the drone goes to the left for a number 180 of patches and then heads south. 180 is a value a bit smaller than the diameter of the long range camera, to make sure all patches are covered. This way, the drone does not check the same patches twice in a row. When it gets close to the downmost patches, it goes to the left for 180 number of patches again and then goes to the north. This process is illustrated in the picture on the right, on the top row and it gets repeated until the drone gets to the leftmost patches. Then it turns to the right and scouts the area in the same way as presented above. This is illustrated on the second row of the picture on the right

Charging

With each tick, the battery of the drone gets decreased by 1. After each tick, the drone checks the distance between its current position and the base, using Euclidean distance. If the distance between its current position and the base is equal to its battery + 10, then the drone goes to the base to get its batteries replaced with already charged ones. Since Netlogo does not have a wait function that makes only the drone to stop and not the entire environment, the waiting of the drone was simulated by increasing the fire 4 times. After the batteries are changed, the drones continues to scout or to go to different locations, which will be explained below.

Going to different locations

The operator at the base can choose locations on the map, which the drone will visit. The operator clicks on a location and sends the coordinates to the drone. The coordinates will be placed in a list called location which contains the x-coordinate of the chosen location, the y-coordinate of the chosen location and the distance between the current position of the drone and the chosen location. That list is then added to another list called destinations which contains the coordinates of all locations indicated by the operator. Basically, destinations is a list containing multiple lists. When a location was sent, the drone stops scouting and goes to the given destination. If another location is sent by the operator, the process above repeats and another location is added to the list destinations. At that point, the distance between the drone and the location is calculated for all locations in the list, and ordered in ascending order by the distance. The distances are updated every time a new location is added or when the drone arrived at the first location from destinations, moment in which that location gets removed and the drone goes to the next location on the list. When the list is empty, the drone goes back to scouting.

If the drone runs out of battery while it is visiting the locations indicated by the operator, it goes to the base to charge and then continues to go to the locations from the destinations list. If the drone spots an anomaly with the long range camera, it goes to the place the anomaly was spotted instead of going to the indicated location. In case there is a fire, the drone will alert the operator.

Protocol

In this section, a detailed look at how project O.W.L. can be implemented in the summer of 2021 is given. Every suggestion made is based on conclusions drawn from previous headings.

The team

The O.W.L. team consists of three members during the first phase of implementation. These members are certified drone pilots with the capability of flying fixed wing drones in line with European regulations. Training such pilots usually takes one week. Some additional training is required to understand the specific tasks that come with the job.

Since the O.W.L project is only active during dangerous weather conditions, when wildfire risk is high, the pilots are only needed a small portion of the year. An inconsistent schedule makes training and employing the right people difficult. Some groups to consider for the program are volunteers, students or anyone available during extreme heatwaves or drought. Most ideally, the team consists of at least one firefighter with drone experience. To this end, an agreement between staatsbosbeheer and the national fire department can be arranged. Both parties will benefit from such a contract.

The drone

Avy Aera

The right drone for the job is a fixed-wing hybrid drone with an electric motor. There are quite some suitable options such as the Avy Aera or the HCS Quantum Systems Trinity. Combining the efficiency of a fixed-wing drone and the mobility of a multi-rotor drone it will be able to survey a larger area for longer at once while also being able to land and take-off almost everywhere. With endurances of up to 60 minutes and ranges of up to 25 kilometers, virtually any area in the Netherlands can be patrolled. The choice for an electric motor was made to reduce noise and also because they are better for the environment.

A payload capacity of up to 1.5 kilograms makes it more than suitable to carry the FLIR Vue Pro R radiometric thermal camera, which weighs around 150 grams. The Vue Pro has a resolution of 640x512 giving the most precise temperature readings. The camera is also attached to a gimbal for complete control and high flexibility. A field of view of 45 by 37 degrees results in an SSR of 0.44:120. From quite a high altitude a relatively small area can still be measured.

Control and communication with the drone will be done via radio frequencies. Depending on the size of the area a choice on which frequencies used can be made. For a range smaller than 10 kilometers a control frequency of 900 Mhz and a data communication frequency of 5.8Ghz is recommended. This gives best video quality. However, an almost clear line of sight is necessary, which means an antenna has to reach above the treeline or other obstructions. For a range larger than 10 kilometers, the data communication frequency is reduced to 2.4Ghz, increasing the range to around 25 kilometers while still able to send decent video quality.

System installation

Ground control station

The flight time for the electric hybrid fixed-wing drone will be around 1 hour. Until a system for automatic battery replacement is created and implemented, an operator has to be on site to manually replace the battery. Since the area is relatively small and easily contained in the range and flight time of the drone, only a main operating base is necessary. Every hour the drone will land and it's battery will be replaced by the on-site supervisor. Ideally, this main operating base will be stationed in the center of the area, to decrease the maximum range necessary for the UAV. If this is not possible due to various reasons, the main operating base can be stationed at the rim of the area. A supply of power and a relatively high radio tower which can remain LoS with the UAV is required. This does not have to be a stationary tower but can be a portable antenna for mobility if necessary.

Average work day

When the project starts, technology for automated drone flight is not regulated. Therefore, the drone needs to be controlled by humans at all times. There are three roles that need to be filled: pilot, camera operator and supervisor. To combat fatigue and monotony, these roles are rotated periodically.

When a person fills the pilot’s role, they are in control of the drones movements. They have access to a standard drone controller and regular video live feedback. On top of this, a map with the drone’s GPS location is also available. They need to follow the predesignated flight path, marked by waypoints on the real time drone location map. The camera is in place to assist with precise movements, such as: take-off, landing, turning and height control. The regular camera feed also doubles as a wildfire detection device. If a fire escalates to the point of visible smoke formation, this camera can assist in long range detection. This is by far the most work-intensive role of the team, requiring drone piloting, large scale maneuvers and smoke surveillance at the same time. Therefore, a maximum time of one hour is recommended.

The second role, camera operator, has less tasks to carry out. They are in control of the zoom, mode and movement of the thermal camera. They are trained to optimize the area under surveillance with clever use of camera movement. They are also the primary means of fire detection. They can detect fires based on live thermal camera feedback. They need to be aware of the camera’s rotation with respect to the drone as well. Because when something that is suspected to be a fire is spotted, they need to inform the pilot to take a closer look. Most of the time, this task consists of evaluating live thermal camera feedback, which can induce inattentiveness, because of its monotonous nature . This is why this role needs to be switched out regularly as well.

The supervisor is the last role in an O.W.L. team. It is their task to more generally keep overview over the entire process. They help with reviewing the drone route and make sure no risky areas get overlooked. They are also responsible for charging the back-up batteries. A supervisor can fill in for another role if a team member needs to take a bathroom break. The time a team member spends as supervisor can be used to eat lunch as well. Because this role has the least work-load by far, it is also in place to mentally refresh the member and allow them to take a break.

As mentioned before, it is advised that a team rotates roles every hour. This allows them to stay alert more often and relieves the workload substantially. This rotation time is also chosen because it coincides with the chosen drone's average battery life. Because piloting the drone involves the most risks and requires optimal attention, the supervisor switches to pilot every hour. This leaves the following rotation: supervisor → pilot → camera operator → supervisor.

Effectiveness of the current system

To get an idea on how effective the drone in the simulation is, we have run 100 resolutions of the simulation. For each test, the drone started out with a fresh battery from the base. The drone uses its own algorithm to determine its flying path. The fire was also reset to a random location on the map. Keep in mind that just to be on the safe side, we have used lower drone flying speed and a lower battery life. Also, the drone could potentially already fly at a higher height. We do this, because the drone can be used more reliably in these settings, thus making the results more realistic.
Even though the simulation only took place in the Loonse and Drunense Duinen, we believe that these results can also be used to accurately predict the succes rate of the drone in other areas in the Netherlands. For the drone, a different area does not change much. The thermal camera is not dependent on the biome on which the fire has started, as of course the temperature for the ignition of a fire is practically always significantly different compared to its surroundings. Regardless, most nature conservations in the Netherlands do not differ that much from one another.

From the 100 runs, without any direct human involvement for the routing, in 60 cases the drone succeeded in finding the fire on time while in 40 cases the fire got too big. The drone is capable of finding the fire on time, though not very reliably. If the team would set up a smarter route for the drone, then the results would be better. Furterhmore, even if the drone does not detect the fire in time, having a drone in the area of a fire is still very advantageous.
In the future, one could experiment with using multiple drones at the same time. From the results, it is clear that, at this moment, the area used for this experiment is too big for one drone to cover reliably. Interestingly, on average, when the drone found the fire, the battery still had 10600 ticks left, while it started with 10800. This means that only 2% of the battery has been used. From this we can conclude that the drone can drain its battery pretty far without needing a recharge, as it does not cost much battery life to find the fire.

Protocol in case of emergency

Fire detected

Once a spike in temperature on the ground is picked up with the drone, the pilot will need to get the drone closer to the site where the temperature was measured. When they can confirm that a wildfire has actually been detected, the fire department has to be called manually. The operators will have to check the size of the fire and the location to inform the fire department. The fire department will also have to know where the closest source of water is. The operators can use the drone while the fire department is on their way to look what lake for example is close to the fire and communicate the location of this water to the fire department as well.

The team member operating the camera can also spot smoke in the air above the nature reserve. When this operator sees it and can confirm it is not from a chimney or a campfire, it will have to be investigated the same way as high temperatures on the ground.

A possibility is to call the fire department immediately after the operators see a big part of the forest in dangerously high temperatures. This has to be decided on the spot depending on how sure the operators are of an actual fire being detected. If the call is made to notify the fire department right away, the exact location is not yet known. The operators could let the fire department know to come to the vicinity of the fire and communicate the exact location once the drone has reached it.

Drone malfunction

The biggest risk the drone faces is collisions. This is possible with either trees and other structures or wildlife such as birds. When the drone pilot tries to get the drone closer to the site of possible wildfire, surrounding may get less of a focus. It is always very important to use the camera for caution of the drone, even when the operators want to get a clear image of the fire.

Besides the drone colliding with surroundings by fault of the pilot, there is also a small possibility of wildlife interacting with the drone. There have been occasions of birds accidentally or intentionally crashing into drones. This is especially the case with hunting bird such as eagles or hawks. Although hawks and other bird of prey have been spotted in the Loonse en Drunense Duinen, the amount of these birds are very low. Their interaction with the drone very unlikely to become a large issue, but there is always a risk.

Loss of camera feed and communication with the drone will not be an issue in the Loonse en Drunense Duinen. The area spans across a length of 10 km at most (google maps approximation). This means that the control frequency of 900Mhz and the data communication frequency of 5.8Ghz will be enough for this area.

Roadmap for the future

As technology improves and regulations become less strict, new innovations become possible. This roadmap highlights changes that are possible in the coming years.

Drone

UAV's are a popular technology and new innovations are made at a fast pace. Project O.W.L. can be expanded accordingly.

Once increased battery sizes or greener and quieter combustion engines have been invented, the flight time of a drone can be greatly increased. Currently only one hour of flight time is possible with an electric motor but this could be increased to 12 or even 24 hours of flight time. Another option which can already be implemented in project O.W.L. is the introduction of multiple drones. However multiple drone teams are then necessary to control the drones. These innovations increase the overall coverage of the area, reducing the risk of wildfires.

As discussed in the chapter on Data communication, LTE communication is a promissing way of data communication, which will greatly increase the range to 100 kilometres and increase the quality of video imagery.

Finally new innovations in thermal imaging technology can increase the resolution and SSR of thermal cameras. More accurate and precise readings can be made from further distances, improving the data and increasing the area that can be patrolled at once.

Degrees of Autonomy

By implementing automatic systems, O.W.L.'s teams size and workload can be greatly reduced. The following implementations are possible with appropriate legal permissions and quality assured software.

Automatic Fire Detection

By analyzing thermal images, the camera operator can spot wildfires. They do this by watching for dangerous temperature values. Software for thermal image analysis already exists. Its implementation and accuracy on wildfires is not yet tested, however. After thorough testing, a feature that automatically sends alarm in case of a wildfire can be implemented easily. At first this feature can be run through a script that directly views the thermal images as they come in. Later, this application can be implemented in the software developed specifically for O.W.L. When software that allows for automated controls is approved, the thermal camera’s movements can be automated as well. While testing this, the thermal camera operator still needs to review the images in real time. But when the autonomous fire detection is proven to be effective, the team’s size can be reduce to two.

Automatic Routing

Allowing the drone to fly to waypoints automatically is the next logical step in automation. Software that plans drone movement is already available, but illegal to use for our implementation. If the right permits are granted, automated drone flight using third party software is a possibility. This allows the role of pilot to become less intensive, reducing their tasks to: take-off, landing, drone location surveillance and smoke spotting. Autopilot can be aborted at any time to take manual controll of the drone in case of emergency.

Automatic piloting

The tasks the pilot has to perform can, in theory, be automated as well. Automatic take-off and landing are possible under stable circumstances. However, this technology is not robust enough to be considered safe to use. This technology will advance in the future. When the time comes when it is considered safe by law, the pilot’s role can be merged with supervisor. This reduces the tasks of the team to a single person, who’s attention is not needed at all times. This single person is responsible for the drones behaviour and location, but managing this has become very simple. Other tasks include reporting to the fire department, reviewing thermal feedback and manually taking control of the drone in case of an emergency.

Communication improvements

Once drones have proven themselves for spotting wildfires, they may possibly create an integration with the fire department directly. Having a separate division be in charge of the drones and call the fire department manually can be improved in the future. Not by dropping the O.W.L. team at the fire department directly but by integrating an improved system of communication. Compared to the O.W.L. operators making their own assessment of wildfires to communicate to the fire department, having the fire department get images and information directly from the drone will be a big improvement.

The fire department may shave off some reaction speed which may limit escalation of fires and the fire department will be able to make their own assessment of the situation and perhaps sent appropriate response teams more often. These response teams can keep getting information from the drone if the fire department has integrated the O.W.L. team. Instead of a call to the fire department from a seperate O.W.L. team, it is more of a united front and the drone can be used for a longer time which can give the response team an advantage.

Appendix

Appendix A: Interview with the fire department

“What are your roles in the fire department?"

Mark: We focus mostly on the outside world. Especially flying preventively. For example, above nature reserves. In the case that happens, the drone can aid you, because the drone can provide more information from above, such as: how are the buildings placed in relation to each other, where are the vehicles, where is the smoke going, are there any people on sight, are there any dangerous chemicals etc.. In that way, a drone offers the 3D perspective that a ground team does not have.

“What do you feel is the biggest problem faced in fire fighting nowadays? What could have the biggest impact on the speed with which fire can be controlled and extinguished?”

Paul: For that I have a question for you. What do you exactly want to achieve? In the Netherlands there are 25 different fire departments and some of them already work with drones. There is also a national drone organization for the fire department, so we already know how to use drones.

“Our plan for now is to find out how drones are currently used and if we could invent a new way to use drones or to improve upon an existing method. We don’t think that drones are capable enough for dousing fires. We are focusing on communication via drones and the 3D perspective that drones offer.”

Paul: Actually, in China and Dubai there are drones that are capable of dousing fires. These drones are of course much bigger and heavier up til 60 or 100 meters high. There are also different ways of using water to douse fires. 10 or 15 years ago a system I think was called ‘3FX’ was created where a helicopter fired a very quick burst of water on a fire. But one could use chemicals to douse fires, or maybe even with sounds. So almost literally, the sky is the limit. It is more a matter of: who is going to design this and who is going to pay for this? In the Netherlands this will probably happen not so fast, since in the Netherlands less money is spent on the fire brigade than in Dubai, for example.

Mark: For basic operations, we already have enough materials to deal with those. For us this is more about the escalation incidents. At this moment, we use drones primarily for gathering information at the place of interest, so that we can create a full picture of what is happening there. Though you would rather want to act at the beginning of the chain; preventing the incident from even happening. So very rapidly being able to detect a starting fire in the aforementioned nature reserve, for example. I think that we can gain the most in this aspect.

“That is indeed a new idea. We can definitely work with that.”

Paul: Also, the Rijkswaterstaat (Dutch governmental organization for infrastructure and such) uses drones for checking the quality of dykes or letting a drone check the quality of the air near factories. Again, the potentials are limitless. It is just a matter of how money you want to put in it.

“Alright. Because of extreme heat and smoke in a burning building, a thermal camera would probably be overloaded to function properly. But imagine that such a camera does exist. Do you think that this will help with the communication between firefighters that are already inside the building and the ones that are still outside? What interesting things could then be seen?”

Paul: Currently, there are remotely controlled drones with a camera attached that already can go inside buildings. We are also working on drones equipped with measuring equipment to measure the concentration of gasses in the air.

“Drones cannot get inside a burning building easily in general. What can drones do before the actual firefighters arrive at the incident?”

Paul: I’ve noticed that you have said a few times now that certain things are not possible. But actually a lot of these things áre possible. A drone can fly into a building. One only needs to design doors that can automatically be used, by sending signals between the door and the drone. We are also working with another team from the TU where we are creating drones that can escort people safely outside buildings.

“Are you concerned with legal restrictions on drones? Obviously firefighters have to follow different rules than recreational pilots. When a new drone is available, can you use it immediately? “

Paul: Of course we can use a drone if it’s legal. If it’s illegal we can’t use it. There are rules in place. In fact, the invitation for the training of new drone pilots for the fire department is open at the moment. If you have the proper training and licence you can start flying drones. In a few areas we cannot use drones at the moment right Mark?

“If drones were to be used, you would need a specialised team. Is this something that is already done? Are you looking for pilots at the moment?

Mark: We already have a few drone teams active at the moment, with flying licenses. These people followed pilot training for drones. As well as the supporting team, they are self supported. And for every incident they can be deployed. So regarding your question, drone teams are already active in the Netherlands.

“I had a question about when drones could be actually used, but it seems that they already are. Are there other alternative methods, apart from drones, that are currently being looked at or tested?”

Mark: Those would be dousing robots and such. Those are currently being researched. Multiple regions already have them, and are testing with them. But drones can also be used for other purposes, like personal transport. The ministry of defence is currently looking into this. You could hang a patient from a brancard under a drone. So you can secure people from heights quicker. But there are also robots you could use for salvage, transporting materials in impassable terrain, those sorts of applications.

“Regarding the drones that are already being used for entering buildings: Are these drones capable of recognising people in distress? How would that work?

Mark: Drones can already do this with a thermal camera. These can thermally distinguish the environment from victims.

“Wouldn’t the camera be overwhelmed by the heat of the environment?”

Mark: Yes indeed, these temperatures, especially in burning buildings with a substantial fire, are radiated everywhere. With thermal cameras you can ask for different thermal images. So you have images with colour or greyscale. When looking at a colour image, everything would look red, so you cannot make sense of it. But when working with greyscale, you can still distinguish persons.

“We imagine that flying close to fires can cause major problems for drones, like melting propellors or dysfunctioning equipment. How is this currently worked around?”

Paul: It depends on what you want to do with the drone. At the moment, it is not a priority to fly the drone directly at the fire. We use these drones to spot people. Or, in early stages, escort people outside. When it comes to dousing, we would rather use a dousing robot (with a camera and hose) than a drone. Drones that are used right now, are not capable of flying through fires. But this is not necessary. We are using them to see where the fire is and what the best plan of action is for extinguishing. They are namely deployed in spaces that are too dangerous to enter, like a big lobby. We can use the zoom function on the cameras to look at the fire closely, and we can estimate distances (by using lidar or radar). So by keeping drones at a distance, there is no need for a heat shield that can withstand 800 degrees. It depends on the drone’s purpose. Right now, we cannot see why a drone would need to fly through fire.

“What are typical temperatures firefighters need to deal with?”

Mark: That is a difficult question, it depends on what is burning etc. A big fire would be 1000 to 1500 degrees. Paul: Of course, we don’t send people in with these kinds of temperatures. Typical temperatures that we can still send teams into would not be 1000 degrees. Mark: Typically we are present when there is a flash-over. With lots of smoke, where everything in the room starts burning. This is typically around 600 degrees. This is still too high, when people are present at these temperatures, we are doing something wrong. So our maximum temperature would be 500 degrees. ...At the ceiling, lower to the ground the temperature is lower.

Daniel: These were all the questions we had come up with, maybe Ruben or Tristan came up with any questions?

Ruben: Since European rules came into effect since this year whether something has changed for you or if you have an exemption when it comes to the use of drones?

Paul: I don't know what the impact of European regulations is, but of course we have to comply with the regulations, which is why our pilots are specially trained. I do not know what the substantive changes are, but we are complying with the laws and regulations. We are working with the legislator to see when we can get an exemption. We do it not for fun but for a social dilemma. And then it seems logical to me that you look together with the legislator when you can get exemption, you must of course be strict. When you fly over an audience, I understand that of course you have to be strict with that. But it is something else if you want to work with a drone on public safety, for example if you are going to fly above riots it is something different than if you are going to fly above an audience at an event in your spare time to make beautiful film images. Of course we will look at where we can get exemption, but we do have to deal with the laws and regulations. My counter-question remains what you want to achieve with this investigation. I say this because we already do a lot with drones in the fire department and it would be nice if you would do additional research and not, so to speak, spend a lot of time in a report and that Mark and I say "yes but, we already have that in Twente, in Twente a drone is already going to leave automatically in case of a fire report. And I already have Utrecht and there it already is" because then you have a report that we say "yes, this may be nice but that doesn't make us any wiser and you might not be any wiser yourselves eventually." Hence, my counter-question about what exactly do you want to achieve with your research.

Daniel: We've already done some research into what kind of drones are used in the fire department but we haven't been able to find much of it and from what I'm hearing now I think we need to look even deeper. But as far as we have found there are only a few drone teams in the Netherlands, I thought we had seen somewhere that there are only 7. I do not know whether we can find exactly to what extent they are now being used, how much information we can find about them. Until now, our plan was to see if we could come up with an innovation and of course we needed more knowledge of what is now being used in the fire department.

Paul: No, I get that. Then it is easiest for you to talk to the national coordinator, Mark Bogdan, Via the website brandweer.nl you can certainly find it. He can tell you all about it. I’m having a hard time wrapping my head around what you could deliver right now, and this is what I mean positively and I mean it mostly realistically.

Daniel: Yes, we understand that. We do not yet know exactly what we want to do and that we still have to do a lot of literary research ourselves.

Mark: What you notice, is that a lot is being put into a piece of exploration and extinguishing with all kinds of systems at the moment. But I think most of the profits can be made, and Paul and I have been in the field for that before, right at the front of the chain. So being able to signal it very quickly and get the profit there. That is also the most difficult thing at the same time. Prevention. We've also been talking to the Department of Defense about it. They want to use these systems in war zones to take over the tasks of patrols. Patrolling, so to speak. And being able to signal about hostilities in time or things like that and immediately respond to that. In fact, the same task could also lie with the fire department, only from a different capacity. So if you already know that you have risk areas somewhere, which Paul just described, and you can fly a drone over there in a certain time and it can send signals immediately to report it to the fire department which saves us a bit of alarm time, road time and deployment right away, which prevents those escalation fires. I think there's a lot of profit to be made there, but it's also the hard part about the story.

Ruben: To what extent is that already being done, flying for prevention?

Mark: I don't think the fire department does it. But there will be other parties that do do something like this preventively, for example, think about events with crowd management where they can just fly a drone above. If somewhere a disturbance is found and a signal is sent, they can immediately send people. You're faster with that than if you constantly have to send teams into the field to keep up. Just something comes to mind. So I think for example an event industry, those kinds of parties that work are already thinking along. Police might already be.

Ruben: By risk areas you mean, for example, nature reserves when it is very dry and warm. Also in urban areas it could also be flown on prevention or, is that a lot more difficult?

Mark: Yes, when you talk about buildings, of course it becomes a lot more difficult, because then you’re talking about systems that can look through something. But then the question is whether that task is for us or whether that task should not be invested with another party. Paul: And that's the question anyway. In Oost 5, in the east of the Netherlands, a project is being worked on to achieve autonomous nature fire exploration, that a drone just flies around there itself. The question is also, is that the responsibility of the fire department or is that the responsibility of the owners. And often it is the responsibility of the owners. Also for the railway sector, then should not be a drone of the fire department flying above to look for leaks, there would be mainly ProRail or something, having to invest in a system that discovers that something is going on there. When it comes to natural fires, the Staatsbosbeheer has to do so. This applies, of course, to all parties. In this sense, we mainly see drones as a tool that we can bring along or can send in advance when we are notified, but building owners or owners of areas etc., they can mainly use drones from the preventive atmosphere. If you look at a port area with a container storage area, it would make sense if [the owners] are the ones who mainly look at whether containers are leaking or if there may be people crawling in or out who do not belong, either human trafficking or crime. So we are also talking about that than, for example, the fire department or the police should do so or, if mainly the owner is primarily responsible for safety and surveillance and crime. And that also applies to event organization, event organizations are, of course, also primarily responsible for ensuring that an event goes well. So they will primarily have to use the drones to see ‘how am I going to make sure everything goes safely’. Because now you already have to, as Mark just said, you have to make people walk around for that now, but you can also use such a drone. Well, that's also possible at a trainway complex, you can put all kinds of gadgets on the ground there, you can also fly a drone around but keeping in mind that as owner he is responsible of course for ensuring safety and preventing nuisance. And if that goes wrong, then the emergency services like us come mainly to contain the emergency, because the prevention is of course primarily up to the owner.

Appendix B: Time log

Week 1

Name (Student number)	Time spent	Tasks
Tristan Deenen (1445782)	6:15h	Meetings (1:30h + 1:15h + 1h), Brain storming (1h), reserach (1:30h);
Jos Garstman(145722)	5:45h	Meetings (1:30h + 1:15h + 1h), Brain storming (1h), research (1h);
Oana Radu (1325973)	6:15h	Meetings (1:30h + 1:15h + 1h), Brain storming (1h), research (1h), wiki entry (0:30h)
Ruben Stoffijn (1326910)	6:45h	Meetings (1:30h + 1:15h + 1h), Brain storming (2h), research (1h) ;
Daniël van Roozendaal (1467611)	5:45h	Meetings (1:30h + 1:15h + 1h), Brain storming (0:30h), research (1:30h);

Week 2

Name (Student number)	Time spent	Tasks
Tristan Deenen (1445782)	10:00h	Meetings (1h + 1h + 0:30h), Talking to firefighter and summarizing that (1:45h), Reading (2:45h), Research (2:15h), Edit wiki (0:45h)
Jos Garstman(145722)	5h	Meetings (1h + 1h + 0:30h), Reading and finding sources (2h), editing wiki(0:30h)
Oana Radu (1325973)	7:30h	Meetings (1h + 1h + 0:30h), Reading(2:30h), Research (2h), Edit Wiki (0:30h)
Ruben Stoffijn (1326910)	5:30h	Meetings (1h + 1h + 0:30h), Reading/Research (2h), Letter (1h)
Daniël van Roozendaal (1467611)	6h	Meetings (1h + 1h + 0:30h), contacting firefighters for interview (1:30h), reading articles (2h)

Week 3

Name (Student number)	Time spent	Tasks
Tristan Deenen (1445782)	4h	Meetings(0:30h + 0:30h), Research(1:45h), Edit wiki(00:30h), Add USE to wiki(00:45)
Jos Garstman(145722)	5:30h	Meetings(0:30h + 0:30h), Research(4h), Edit wiki(00:30h)
Oana Radu (1325973)	5h	Meetings(0:30h + 0:30h), Research(4h)
Ruben Stoffijn (1326910)	5:30h	Meetings(0:30h + 0:30h) Research (4:30h)
Daniël van Roozendaal (1467611)	5h	Meetings(0:30h + 0:30h), Research (4h)

Week 4

Name (Student number)	Time spent	Tasks
Tristan Deenen (1445782)	6:30h	Meetings(1h + 1h), Interview and transcription (3h), Research simulation (1:30h)
Jos Garstman(145722)	6h	Meetings(1h + 1h), Research(2h), Editing wiki(2h)
Oana Radu (1325973)	8h	Meetings(1h + 1h), Research(3:30h), Research Simulation(2h), edit wiki(0:30h)
Ruben Stoffijn (1326910)	8h	Meetings(1h + 1h), Interview + Transcription (3h), Research(2h), Introduction (1h)
Daniël van Roozendaal (1467611)	7h	Meetings(1h + 1h), Interview + transcription (3h), Wildfire research (2h)

Week 5

Name (Student number)	Time spent	Tasks
Tristan Deenen (1445782)	8:30	Meetings(1h + 0:45h), simulation(6h), edit wiki(0:45h)
Jos Garstman(145722)	10:15h	Meetings(1h + 0:45h), Research and editing wiki[Drone functionalities](5h), Research and editing wiki[Drone functionalities/Drone types](3:30h)
Oana Radu (1325973)	11:15h	Meetings(1h + 0:45h), Simulation(7:30h), edit wiki(1h)
Ruben Stoffijn (1326910)	12:45h	Meetings(1h + 0:45h), User requirements (10h), Editing wiki(1h)
Daniël van Roozendaal (1467611)	10:45h	Meetings(1h + 0:45h), Wildfires(5h), Detection in the Netherlands(3h), Editing wiki(1h)

Week 6

Name (Student number)	Time spent	Tasks
Tristan Deenen (1445782)	12:30h	Meetings (1h + 2h), simulation (8h), research accurate values for simulation (1:30h)
Jos Garstman(145722)	12:00h	Meetings (1h + 2h), Research and editing wiki Wednesday (4h), Research and editing wiki Sunday(5h)
Oana Radu (1325973)	12h	Meetings (1h + 2h), Simulation (8:30h), Editing wiki (0:30h)
Ruben Stoffijn (1326910)	14h	Meetings (1h + 2h) , Drone regulations touch-up(2h), responsibility research + writing (4h), protocol brainstorming +writing (5h)
Daniël van Roozendaal (1467611)	13h	Meetings (1h + 2h), Taskforce Natuurbranden (5h), Captured Data (3h), Fire spreading speeds (1h), Editing wiki (1h)

Week 7

Name (Student number)	Time spent	Tasks
Tristan Deenen (1445782)	19:45h	Meetings (1:30h + 2:15h + 0:30h + 0:45), simulation (10:00), readjust accurate values (1h), presentation(3h), edit presentation (0:45h)
Jos Garstman(145722)	19:30h	Meetings (1:30h + 2:15h + 0:45), Wiki editing(10h), Making slides(2h), Recording presentation(3h)
Oana Radu (1325973)	21h	Meetings (1:30h + 2:15h + 0:30h + 0:45), Simulation(9:30h), Presentation(3h), Test simulation(2h), Edit video(1:30h)
Ruben Stoffijn (1326910)	17:30h	Meetings (1:30h + 2:15h + 0:45), protocol team+roles+roadmap (7h), presentation(4h), editing wiki (2h)
Daniël van Roozendaal (1467611)	16:30h	Meetings (1:30h + 2:15h + 0:45), protocol emergencies + malfunctions (6h), communication improvements (2h), presentation (4h)

Week 8

Name (Student number)	Time spent	Tasks
Tristan Deenen (1445782)	15:30h	Presentation(2h), Meetings(1h + 1h + 1h), Write about the simulation (7:30h), Final evaluation(1:30h), Edit wiki(1:30h)
Jos Garstman(145722)	11h	Presentation(2h), Meetings(1h + 1h + 1h), Editing wiki(5h), Final changes(1h)
Oana Radu (1325973)	12h	Presentation(2h), Meetings(1h + 1h + 1h), Write about the simulation(6h), edit wiki(1h)
Ruben Stoffijn (1326910)	12h	Presentation(2h), Meetings(1h + 1h + 1h), Editing wiki(5h), Final evaluation/comments (1h), final touches(1h)
Daniël van Roozendaal (1467611)	12h	Presentation(2h), Meetings(1h + 1h + 1h), Editing wiki(6h), Final evaluation(1h)

References