View source for PRE2017 3 Group 17 - State of the Art

State of the art report for the project of [[PRE2017 3 Groep17]].

== Aerial Land Surveying ==
Traditionally aerial surveys of forests or archaeological sites were done either by using publicly available satellite images or by flying over the area with a plane. More recently however researchers have started using Unmanned Aerial Vehicles or Drones for this purpose. Early variants of these drones were hacked together from model airplanes and other DIY equipment to make them autonomous<ref name="ConservationDrone">[http://journals.sagepub.com/doi/abs/10.1177/194008291200500202 Dawn of Drone Ecology: Low-Cost Autonomous Aerial Vehicles for Conservation (2012]</ref> but as drones have become more popular and readily available, researchers have started moving to off-the-shelf equipment and software<ref>[https://www.researchgate.net/publication/299503960_Testing_UAV_drone_aerial_photography_and_photogrammetry_for_archaeology Testing UAV (drone) aerial photography and photogrammetry for archaeology (2016)]</ref>. Although there are still cases in which drones might not be the best solution, the resolution of these drone surveys is far superior to the traditional aerial surveying methods<ref name="ConservationDrone"/><ref>[https://link-springer-com.dianus.libr.tue.nl/article/10.1007/s41636-017-0069-4 Drone Mapping and Photogrammetry at Brandon House 4 (2017)]</ref>

== Payload Issues ==
Drone payload (how much weight they can carry) can become quite an issue when wanting to use them for lifting the (often) heavy objects on construction sites. Inspecting what commercial sellers currently sell shows that most cheap and small sized drones have a limited carrying capacity of around 1kg max, which is obviously not enough to be used on construction sites.

There are however several niche heavy payload drones for sale by these same sellers, able to lift a maximum of 10kg while still being relatively small sized. This is however also shown in the price-tag as their prices range from €1,000-€10,000 euros!
<ref>[http://3dinsider.com/drone-payload/ How much weight can a drone carry?; 3D INSIDER (2017)]</ref>
<ref>[http://www.dronesglobe.com/guide/heavy-lift-drones/ 7 DRONES THAT CAN LIFT HEAVY WEIGHTS; Drones Globe (2017)]</ref>

In late 2017, Russian researchers developed what is known as the ‘hulk’ drone. This drone has a massive 181kg payload (amongst other high statistics such as maximum flight speed or flight time). The size, 2.2 by 5.2 meters, however is an issue for this project’s specific problem, as this doesn’t allow the drone to fly indoors in most buildings. There may be other applications for this drone, such as them being able to replace a crane for delivering panels for a roof, though that’s not the goal of this project as it’s specifically about indoor flight.
<ref>[http://www.dailymail.co.uk/sciencetech/article-5115609/Russia-unveils-SKYF-heavy-lift-drones.html The Russian 'hulk' drone that can lift a 400lb payload and fly for up to eight hours; MailOnline (2017)]</ref>

The low payload could be solved by using multiple drones lifting the same object, utilizing their individual payloads as a whole to be able to lift heavier objects together. Software for controlling such groups of drones lifting one object already exists and has already been patented (W. So et al.).<ref>[https://patents.google.com/patent/US9043052B2/en System and method for multiple vehicles moving a common payload; Wilfred So, Justin Eichel, Linda Vu, Peter Szabo (2015)]</ref>

The patent describes that the drones form a swarm controlled by a driver or pilot station, of which the latter can also be situated on one of the drones itself. A payload destination can be entered remotely at the pilot station, which is then processed into multiple individual waypoints and are then communicated to the other drones.

C.A. Rabbath describes a Final State Machine for multi-vehicle airlift, although it does mention that follow-up research is necessary since the FSM was tested without an actual payload.
<ref>[https://link.springer.com/article/10.1007/s10846-012-9692-7 A Finite-State Machine for Collaborative Airlift with a Formation of Unmanned Air Vehicles; C.A. Rabbath (2012)]</ref>

== Drone movement and control ==
Drones need a way to be controlled, so they can more efficiently perform the task they were designed to do. Since this project revolves mostly around getting visual feedback from drones, articles with that subject were sought. A recent patent about Automated drone systems <ref>[https://patents.google.com/patent/US9864372B2/en Automated drone systems (2015) ]</ref> in automated security systems shows that companies are already busy trying to achieve with one drone what this project tries to achieve with multiple. Besides that it shows that there are systems capable of navigating one drone to a desired location, and that it has a smart way to scan surfaces.

This project has mostly been focussed on airborn drones. However, the submarine drones used for the search of the MH370 jet show that the project can easily be applied to similar underwater activities. <ref>[https://ac-els-cdn-com.dianus.libr.tue.nl/S0262407914607545/1-s2.0-S0262407914607545-main.pdf?_tid=8320553a-0c4f-11e8-9b0e-00000aacb35f&acdnat=1518039808_a38e434108e2b128df11e71965e51fa2 Submarine drone dives into search for MH370 jet (2014) ]</ref>

Instead of avoiding collisions between drones and between drones and other objects at all costs this article proposes a collision resilient drone. This drone should survive more crashes and is still able to operate like a "normal" drone. It does this with rings around a drone body and a gimbal system allowing it to keep moving even when it hits a wall.<ref name="crashresdrone">[http://onlinelibrary.wiley.com/doi/10.1002/rob.21495/epdf A Collision-resilient Flying Robot (2014)]</ref>

== Localiziation ==
Outdoors drones can localize using GPS and the like but indoors where this signal is a lot less accurate so other methods have to be used. Research<ref>[https://pdfs.semanticscholar.org/f188/873bb0ecedf323bbd518e3605afbb7d3bf2f.pdf Simultaneous localization and mapping with the AR.Drone (2012)]</ref> has already been done into this problem and has found ways to navigate drones using only their camera and accelerometers. This research has been able to get drones to autonomously fly a parcour using no outside sensors/beacons or external processing power. The model and algorithm in this research are even able to traverse over obstacles and staircases.

== Swarm Technology ==
Two “cleaning” algorithms for swarms of robots will now be discussed. It explains how these algorithms can be used as “hunting” protocols to, for instance, find lost humans. The first of the two algorithms, Parallel Path Search, works on rectangular areas only and requires them to have no “leakage”, but is computationally quite light. Leakage is a term used to describe the phenomenon of dirt leaking from dirty sectors to clean sectors or, closer to the example, the opportunity for a fugitive to escape when gaps between drones are too large. The second, the SWEEP protocol, on the other hand, will work on any non-disconnected area and can work around "leakage", but is computationally quite expensive at times and requires an overview of the entire area. The DDDAS framework uses the best of Parallel Path Search and the SWEEP protocol to create a better framework, that also takes potential loss of drones into account. All in all, some very interesting algorithms already exist for the theme in mind. <ref name="SwarmControl">[https://ac-els-cdn-com.dianus.libr.tue.nl/S1877050913005796/1-s2.0-S1877050913005796-main.pdf?_tid=97585fa4-0be9-11e8-a6b1-00000aacb35f&acdnat=1517996033_0b095911bc2b3916dd0de0e660e6fb05 Swarm Control of UAVs for Cooperative Hunting with DDDAS (2013) ]</ref>

Another article found describes a framework for multi-agent research. The platform used for the development of this framework were low-cost quadcopters. The demonstration of the platform on environment exploration and collision avoidance showed that the platform is decent. The largest problem is data loss when using wireless communication over larger distances. This means that the platform still needs improving to make it more robust, consistent and reliable. <ref name="MultiAgentEnvironmentExploration">[https://link-springer-com.dianus.libr.tue.nl/content/pdf/10.1007%2F978-3-319-10401-0.pdf Multi-agent Environment Exploration with AR.Drones (2014) ]</ref>

A recent study has also researched the possibility to train models based on fluid dynamics. <ref>[http://ieeexplore.ieee.org.dianus.libr.tue.nl/document/6407937/ Swarm coordination based on smoothed particle hydrodynamics technique (2013) ]</ref> In this study a model was created (based on fluid dynamics) upon which the individual robots based their movement. The advantages and disadvantages of this approach is that the robots choose the same path a fluid particle would.

This brings with it a couple of advantages. The fluid particles (and thus the swarm of robots) always choose the path of least resistance. Every fluid particle and thus the robot behave the same as every other particle. This means that the robots do not need to be numbered, and thus is the code for every robot the same. The nature of particles also implies that the model makes use of a decentralized architecture, and that the approach is robust to the dynamic deletion and addition of new agents.
The study also mentioned a couple of disadvantages associated to this approach. First of all a lot of parameters are needed for the model to be used. If these parameters are not correctly adapted at the task at hand deadlocks may occur. Another disadvantage this approach brings with it is the fact that every robot needs a complete map of the environment it is going to function in. This flaw is inherent to the decentralized structure, but is present nonetheless. Lastly, each robot needs information about its surroundings which are normally provided (in the case of the particle) by physics. The needed info is the range, bearing, and velocity of neighboring robots. The writers do remark that the requirements are consistent with multirobot path planning approaches, and are thus not necessarily disadvantageous to this particular approach. 

When implementing this system on airborne drones, however, another issue occurs. Airborn drones create airflow which influences other drones. In the case of 3D model generation using airborne swarms, three drones where already creating enough turbulence, and thus instability to render them useless for the purposes of this project.

== Sensors and analysis methods ==
The first article belonging to the category presents a list of different sensors and camera's that are currently available for drone flight and sectors where they are used often. These consist of, but are not limited to, accellerometers, cameras (both infrared and normal) and GPS.<ref name="SensorSolutions">[https://search-proquest-com.dianus.libr.tue.nl/docview/1792777729/fulltextPDF/9C3E44230F584C54PQ/1?accountid=27128 Sensor solutions play critical roles in enabling innovation in drones]</ref>

Another article which was analysed discovered shows the development of imaging spectroscopy in multi-temporal environments using a multitude of drones. In layman words, a way to perform analysis on a series of high-resolution photos that have been taken from different directions. This technique is quite promising and seems to perform equally to or better than other techniques.<ref name="ImagingSpectroscopy2D">[https://www-sciencedirect-com.dianus.libr.tue.nl/science/article/pii/S0034425717305072 Multi-temporal high-resolution imaging spectroscopy with hyperspectral 2D imagers – From theory to application (2018)]</ref>

A third article found discussed the evolution of photogrammetric and remote sensing technologies in the field of Unmanned Aerial Systems. It shows that drones are ready for use, but that large regulatory issues still have to be dealt with. However the European Union has hatched a multi-annual plan to integrate the use of drones for mapping, which would remove most current issues. It also shows that the navigation systems are generally ready, but improvement is still possible. Sensors have been developed to the point that they are small and lightweight enough to be used on drones. Additionally, the market of interested parties is growing with the day. In conclusion: additional research and technologies are definitely required and are being conducted. There will definitely be more new UAS technologies and applications in photogrammetry and remote sensing in the coming years.<ref name="UASPaRS">[https://www.sciencedirect.com/science/article/pii/S0924271614000501 Unmanned aerial systems for photogrammetry and remote sensing: A review (2014)]</ref>

A fourth article about the analysis of the images made by the drones. Using a Fuzzy unordered rule induction algorithm the accuracy of detection of objects and type of land usage went from around 70% for the currently used methods to 90%. This means that things like analyzing things like forests or dunes can definitely be done using drones. But this analysis does still cost quite some time and computing power so further research may decrease these costs.<ref name="FURIA">[http://www.tandfonline.com/doi/full/10.1080/01431161.2016.1277043 Drone-based land-cover mapping using a fuzzy unordered rule induction algorithm integrated into object-based image analysis (2016)]</ref>

Another article about analysis of the footage captured by drones discusses ways to analyze crowds. This article talks more about what could be possible then what is possible at the moment using already established object recognition algorithms. These algorithms could be extended to capture things like the flow of a crowd and potential spots where people are slower which could indicate trouble.<ref name="crowdsurvey">[http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5562657&tag=1 Crowd Analysis Using Computer Vision Techniques (2010)]</ref>

Some of the ideas of the crowd analysis have already been implemented with simulation of other real life processes like heat spreading or liquid flowing. The algorithms which use these can be used to map the flow of a crowd and spot places where lots of interaction may occur which can lead to disorder.<ref name="crowdliquid">[http://ieeexplore.ieee.org.dianus.libr.tue.nl/stamp/stamp.jsp?tp=&arnumber=7532547 Crowd behavior identification (2016)]</ref>

== Current usage of Drones in Construction ==
Drones are already being used/ planned to used in and around construction sites. For example to detect unwanted people entering the site and monitoring the construction workers as written about in this article<ref name="droneswatchconstrcutionworkers">[https://search.proquest.com/docview/1732075702?OpenUrlRefId=info:xri/sid:wcdiscovery&accountid=27128 Construction Drones (2015)]</ref>

Siebert et al. describe how drones can be used for surveying construction sites, such as construction of a road, relatively efficiently. They discuss things such as hardware, photogrammetric data processing, and error analysis, which are all necessary for the surveying drones. It is mentioned how the UAVs can perform a high-quality scan, though it is also mentioned that the area to scan needs to be obstacle-free, as these obstacles negatively affect the drone’s performance and result for the need of manual intervention.
<ref name="dronesurveying">[https://www.sciencedirect.com/science/article/pii/S0926580514000193 Mobile 3D mapping for surveying earthwork projects using an Unmanned Aerial Vehicle (UAV) system; Sebastian Siebert, Jochen Teizer; (2014)]</ref>

== Indoor navigation ==
To be able to go from point A to point B, any system will need to find a path between those two points. For indoor navigation this means finding a path through rooms and corridors while only going through doorways. To do so one could stick to one wall, similar to how one would solve a maze, but this would be rather inefficient. Better would be to use a graph based model<ref>[https://link.springer.com/content/pdf/10.1023%2FB%3AGEIN.0000034820.93914.d0.pdf A Spatial Access-Oriented Implementation of a 3-D GIS Topological Data Model for Urban Entities (2004)]</ref> on which to use traditional graph based path-finding algorithms or using a door-to-door approach.<ref>[http://www.isprs.org/proceedings/2011/Gi4DM/PDF/OP05.pdf A "Door-To-Door" Path-Finding Approach For Indoor Navigation (2011)]</ref>. The benefit of using a graph based method is that efficient algorithms exist<ref>[https://link.springer.com/content/pdf/10.1007/BF01386390.pdf A Note on Two Problems in Connexion with Graphs (1959)]</ref> to do the path-finding but the downside is that this may lead to detours if the graph is too abstract. The door-to-door approach is a more difficult way to find paths but does provide more natural paths. It does, however, not take into account having multiple floors. 

Another important part of indoor navigation is indoor positioning. The existance of BlueJay <ref name="blue">[https://www.bluejayeindhoven.nl/about-the-drone/ Blue Jay]</ref> shows that this problem can be solved with the use of indoor positioning algorithms and object recognition.

== Ethics ==
During this project the possibility of drones scanning areas is researched, however this has no meaning if it is ethically unjust or even illegal to do so. The technological advancement of these flying objects has already sparked some debate on whether the law needs to be changed, and if so by whom. <ref>[https://search-proquest-com.dianus.libr.tue.nl/docview/1325836102/fulltext/F3ABBABE77174401PQ/1?accountid=27128 Drones In Domestic Settings Spark Debates About Privacy Often Equipped With Cameras Law enforcement uses the machines to find missing persons as well as crooks]</ref>

A few ethical papers were analysed to get the state of the art with respect to ethics on this subject. Unmanned aerial systems: Consideration of the use of force for law enforcement applications claims that the use of force by a UAV is allowed provided that the amount of force is not excessive. This can be achieved through the codification of a decision making model for law enforcement engagement. Since the purpose of the drones used in this project are solely capable of surveillance, they are always within the maximal allowed use of force in any situation. With regard to privacy the article proposes to obtain a warrant when it is expected that privacy will be intruded upon. <ref>[https://ac-els-cdn-com.dianus.libr.tue.nl/S0160791X13000900/1-s2.0-S0160791X13000900-main.pdf?_tid=fa49aa92-12f5-11e8-93d2-00000aacb35e&acdnat=1518771011_d3aa13eac34d1a1528103d1ab5feac07 Unmanned aerial systems: Consideration of the use of force for law enforcement applications (2013) ]</ref>

All tasks drones are expected to be doing during this project would normally be done by helicopter or manual inspection, so in that case the drone just becomes a tool for a task. The underlying ethics debate is then not about the drones anymore, but about the task, which is outside the scope of this project.

== References == 
<references/>