PRE2018 3 Group17

Group members

Group Members	Student nr.
Diederik Geertsen	1256521
Cornelis Peter Hiemstra	0958497
Joël Peeters	0939193
Benn Proper	0959190
Laila Zouhair	1260529

All Robot Ideas

Below are all ideas that were thought up for the robots everywhere project, they are ordered by the order in which they were thought up. The final idea that was elaborated on was the disaster observation drones.

• Breakfast bot
• Robot to remove microplastics from water
• Clothes folding robot
• Building guidance robot
• Disaster observation drones

Problem Description

The search for survivors after a disaster is an extremely time sensitive matter. Localisation and retrieval of victims should occur in an efficient manner, but due to the nature of the disasters this is not always possible. Examples of these include the damage of roads which hinder rescue teams that are moving towards the survivors, due to damage to the environment survivors cannot be found easily if they are stuck inside of a building. A system must be devised that can help rescue teams with finding these survivors and routing an efficient course towards them.

This system should ideally be usable in the following disasters:

• Forest fires
• Earthquakes
• Floods
• Hurricanes or other weather related disasters
• Nuclear disasters
• (Large Scale) Terror attacks

These disasters each have their own requirements for proper functioning, luckily these requirements can be narrowed down to being able to avoid collision with objects, withstand high and low temperatures and being resistant to rough weather. The system should also be able to move past small crevices while avoiding collision.

Objectives

The most important aspect of the design is that is is able to identify the roads and is able to find survivors. For the first point, the system should analyse the roads and check if they are not obstructed, it can then mark these roads on a map for easy access. It should also be able to find survivors in a given disaster area, this means both inside buildings and outside buildings. It can then place these onto a map and can therefore constuct a sort of heat-map of where the survivors are located, the locations with a lot of survivors can be called high priority zones.

This information can then be combined to construct an ideal plan of action. Using the road data it constructs the quickest route to the survivors. It also balances the high priority zones and closest survivors. This eventually ensures an optimal path through the given disaster area where the most people are rescued in the shortest amount of time. It should also collect this data in an efficient manner which ensures a high speed of the collection of data. It should also be able to function in a high number of disasters, ideally it should function in any type of disaster.

Solution

The entire system consists of four components:

• Finding the roads
• Finding survivors
• Efficient searching
• Optimal path for rescue operations

All of these could be projects of their own, it was therefore decided that the focus should be placed on the efficient searching aspect of the design, and testing this search by finding survivors. If these components are finished then the other tasks will also be attempted, depending on how much time is left.

Finding the survivors and placing them on a map should be the easiest part of the project, as it should only have to identify the survivor and then mark their coordinates. It should be noted that image recognition is not used for this, it will use a chance based system on if it does indeed see a survivor at a given position. This was due to image recognition being another huge part of the project and would remove focus from the specific problem. The chance based aspect was chosen to simulate the fact that image recognition might fail.

The second component is the efficient searching for roads and survivors. This is done by dividing the search strategy into two groups. One group focusses on the search for roads, and the other focusses on finding survivors. These both work in clusters of drones which will take into account where other drones are searching so they don't check the same area twice. This is the main focus of the project, and if this works sufficiently then the remaining sections will also be developed further.

Both of these components will be implemented with drones. This is due to the fact that drones can move through the air, therefore ignoring most of the bad infrastructure after a given disaster. They are also able to move through small crevices which allow them to search inside of buildings with minimal trouble. Due to the nature of them flying there will also be a smaller chance that they cause additional damage because they accidentaly hit a weak point in a construction.

Requirements

The proposed solution when finding survivors and the cluster of drones for efficient searching both have their own requirements that should be incorporated into the design.

The total list of requirements is as follows:

• The survivors can be found through vision
• The survivors can also be found by checking high concentrations of wireless transmission in an area
• It should be able to mark the people on a map
• It should work autonomously
• The drones should not hit each other during flight
• The drones can be sent to a disaster area soon after the disaster hits
• They should check different areas with as little overlap as possible
• Individual drones should be precise enough that they can enter buildings through small holes
• The entire map can be divided into sectors, with clusters of drones being assigned to a single sector at a time
• After finishing a sector, the cluster will move to an unexplored sector
• The drones can change their target if they have searched for their targets everywhere. An example would be searching for survivors after all the roads have been mapped

Some of these can be applied to both the cluster of drones and finding the survivors.

Approach

The approach to solving this problem is simple, first, a thorough literature study will be conducted that will be used to determine the state-of-the-art regarding drone technology as well as their current use in natural disasters and regarding the current other methods of observation during natural disasters and the problems and restrictions associated with these methods. These are then neatly summarised into different sections based on their relevance and subject matter.

Based on this information, an optimal observation strategy for a certain set of defined scenario's will be designed, based on the abilities of drones and maximizing the amount of new information a single drone can deliver. To provide evidence of the working of this strategy, a simulation will be made that shows how a network of drones would operate in such a disaster area.

Depending on if enough time is left for the project, either the other two components will be modelled, or a small scale test setup is built to test the system in reality. The final report will exist of a literature study, an explanation of the optimized observation strategy, and the simulation. It will also include an expansive explanation of the choices made during each part of the project.

Planning

Week 1

Problem-statement and objectives (Cornelis and Benn)
State-of-art (Every member provides at least five sources)
Users and their requirements (Diederik)
Approach, planning, milestones, and deliverables (Laila and Joël)

Week 2

Updated problem description (Benn)
Concrete planning for project (Benn, Laila)
Analysis of literature sources (Joël)
Restructure Wiki (Benn)
Requirement analysis (Benn)
Write down sources in APA style (Laila)
Update wiki (Laila, Benn and Joël)
Simulation methods (Diederik and Cornelis)

Week 3-6

Observation strategy
Work on simulation
Checking the RPC’s
Analysis of decisions made for the simulation and update if needed
Update the wiki
Literature study
Use cases

Week 7

Finalize simulation
Prepare presentation
Finalize the wiki

Week 8

Presentation
Hand in report

Week	Tasks
1	Problem-statement and objectives (Cornelis and Benn)	State-of-art (Every member provides at least five sources)	Users and their requirements (Diederik)	Approach, planning, milestones, and deliverables (Laila and Joël)
2	Updated problem description (Benn)	Concrete planning for project (Benn, Laila)	Analysis of literature sources (Joël)	Restructure Wiki (Benn)	Requirement analysis (Benn)	Write down sources in APA style (Laila)	Update wiki (Laila, Benn and Joël)	Simulation methods (Diederik and Cornelis)
3	Observation strategy	Analysis of decisions made for the simulation and update if needed	Use cases	Simulation
4-6	Observation strategy	Work on simulation	Checking the RPC’s	Analysis of decisions made for the simulation and update if needed	Update the wiki	Literature study
7	Finalize wiki	Prepare presentation

State of the art

It is important to first find out what has already been done on the subject matter. This is done by looking at the state of the art research done for these rescue drones. These can then be used to either help develop the proposed solution, or to use research as an additional component that works in tangent to the solution.

Existing components of the design

The fact that drones have grown massively in popularity over the past decade is clear, but scientists have shown increased interest in drone technology as well [11]. Drones and other UAVs (Unmanned Aerial Vehicles) are now used in a wide variety of applications including the making of movies, surveillance and inspection of industrial structures that take up large amounts of space (oil pipelines, train tracks), mapping geological structures and providing food and medical relief to hard to reach areas [10]. The Delft University of technology recently developed an ambulance drone which carries several useful pieces of equipment over to the specified location, so people can get a head start on helping the person in need [16].

Understanding of how to effectively use drones in different kinds of applications is increasing rapidly, and the subject of using drones in disaster areas is no exception. Research shows that as drones became more affordable, and their technology more advanced, they are becoming increasingly suitable for implementation into disaster areas. Even if rescue personnel are not adept at using the technology, it still manages to increase their efficiency [4]. For instance, the usage of drones as cellular beacons in case cell towers no longer function is being investigated [14], and drones are being used for support and observation, including providing information about the development of forest fires, and information about collapsed buildings [5].

One part of the problem of using drones in disaster areas, namely the autonomous analysis of the images captured by the drones, has been studied extensively. Systems have been developed for detecting potential obstacles for the drones [13], recognizing to what degree a building has collapsed [4], or how to recognize different types of forest fires [15]. Similar systems exist using observation from space by satellite, but these methods often lack the resolution required to get all the information required [17]. The recognition of humans from these camera images has also been studied. This is done either by teaching a system to recognize humans from a set of test images [24], or by comparing heat signatures [26]. Since these technologies are already quite well researched, we will focus on other parts of the project, namely optimizing the search strategy of the drones.

Useful resources for the design

• 1 Talks about how drones can autonomously find survivors by scanning the environment. They offer a high potential for fast and efficient response during a rescue mission. What should the drone do to help the survivors. Needs to observe its environment to avoid a collision.
• 7 Talks about the feasibility of a multi-tier drone architecture over single tier drones in terms of efficiency. This increases efficiency and reduces path loss.
• 12 Mainly talks about how paths are found for drones to follow. and how trajectory planning works, uses decision making and direction of target given a path to deciding what to do.
• 14. Talks about the usage of drones as cellular network beacons in cities after some calamity. Presents a stochastic model to predict how many drones are necessary for a given situation. We could do something similar for our number.
• 18. Proposes HAC-ER, a system for cooperation between information-gathering agents and humans in disaster regions. Shows great promise, problems mainly arise due to airports not easily allowing UAVs in their active airspace.
• 22. Article discusses the benefits and drawbacks of two different communication methods for drone swarms, also explains them. Very useful and relevant for later stage of our project.
• 23. Article acts as an example of how to effectively set up a communication network for a 'swarm' of agents, how to get them to perform tasks. Super useful, but hard to follow.
• 25. Design for a fully autonomous/wireless drone charging station. Useful if we want to include a charging station in our strategy.

Complementary sources

• 2 Discusses failure of a drone system, espionage due to hacking, and autonomous finding of survivors. (Weinig text om er meer over te zeggen)
• 8 Discusses the useful sections of implementing drones in rescue scenarios, as well as how to manage certain aspects of it. It gives a summary of various communication aspects and issues related to their deployment.
• 9 Discussion on why opportunistic networks aren't more common in todays world
• 11 Optimization approaches for different civil applications of drones and characteristics of those types of drones, drones are extremely versatile and new uses are always found for them.
• 20. Article discusses general ethicalness of CCTV surveillance. Concludes that partially automated data analysis from these systems is more ethically preferable to manual analysis. Hard to relate to our case specifically, but could spark the discussion of ethicalness of our system.
• 21. Article discusses ethical feasibility of facial recognition systems. Seems unrelated to our project entirely.

Inaccessible sources

• 3 Presents a vision where the drones provide wireless communication between survivors and cellular infrastructure. (Geen toegang tot volledige artikel)
• 19. This article is not accessible using a TU licence. Abstract talks about integrating calculations for path planning between different scales of a system (i.e. destination of each agent vs. not crashing into each other etc.).

Total list

• 1 Talks about how drones can autonomously find survivors by scanning the environment. They offer a high potential for fast and efficient response during a rescue mission. What should the drone do to help the survivors. Needs to observe its environment to avoid a collision.

• 2 Discusses failure of a drone system, espionage due to hacking, and autonomous finding of survivors. (Weinig text om er meer over te zeggen)

• 3 Presents a vision where the drones provide wireless communication between survivors and cellular infrastructure. (Geen toegang tot volledige artikel)

• 4 Drones become more affordable and the technologies become more advanced, this makes it increasingly more suitable to implement into real disasters. Talks about implementation requirements and about how drones increase the efficiency of rescue personnel even if they are not adept at using it.

• 5 Applications of drones in different kinds of disasters, floods, earthquakes, forest fires and nuclear disasters

• 6 The use of MIMO for communication between drones (Extreme list of equations basically saying that it shows potential)

• 7 Talks about the feasibility of a multi-tier drone architecture over single tier drones in terms of efficiency. This increases efficiency and reduces path loss.

• 8 Discusses the useful sections of implementing drones in rescue scenarios, as well as how to manage certain aspects of it. It gives a summary of various communication aspects and issues related to their deployment.

• 9 Discussion on why opportunistic networks aren't more common in todays world

• 10 General applications for drones

• 11 Optimization approaches for different civil applications of drones and characteristics of those types of drones, drones are extremely versatile and new uses are always found for them.

• 12 Mainly talks about how paths are found for drones to follow. and how trajectory planning works, uses decision making and direction of target given a path to decide what to do.

• 13 Creates a solution for detecting natural obstacles such as trees, and proposes a type of sensor for this.

• 14. Talks about the usage of drones as cellular network beacons in cities after some calamity. Presents a stochastic model to predict how many drones are necessary for a given situation. We could do something similar for our number.

• 15. Discusses the differences in effectiveness between different kinds of camera techniques used in forest fire observation. Concludes that these camera\analysis techniques still have trouble distinguishing between forest fires of different types of foliage. Improvement on this could greatly improve the accuracy of prediction of the speed at which fire spreads.

• 16. Proposes usage of a network of drones, but does not provide further information on this network. Shows that drones are capable of carrying numerous pieces of useful equipment over longer distances when well-designed.

• 17. Article discusses usage of fire detection algorithms from space, concludes that these detection methods are quite accurate and mature except for small or relatively cold fires. Reason why our project may be very useful.

• 18. Proposes HAC-ER, a system for cooperation between information-gathering agents and humans in disaster regions. Shows great promise, problems mainly arise due to airports not easily allowing UAVs in their active airspace.

• 19. This article is not accessible using a TU licence. Abstract talks about integrating calculations for path planning between different scales of a system (i.e. destination of each agent vs. not crashing into each other etc.).

• 20. Article discusses general ethicalness of CCTV surveillance. Concludes that partially automated data analysis from these systems is more ethically preferable to manual analysis. Hard to relate to our case specifically, but could spark the discussion of ethicalness of our system.
• 21. Article discusses ethical feasibility of facial recognition systems. Seems unrelated to our project entirely.

• 22. Article discusses the benefits and drawbacks of two different communication methods for drone swarms, also explains them. Very useful and relevant for later stage of our project.

• 23. Article acts as an example of how to effectively set up a communication network for a 'swarm' of agents, how to get them to perform tasks. Super useful, but hard to follow.

• 24. Article talks about the recognition of humans in all resolutions of IR-photo's. Uses a set of learning pictures to teach the system how a human looks. Very good at finding humans, so state-of-the-art for detecting humans can be defined using this.

• 25. Design for a fully autonomous/wireless drone charging station. Useful if we want to include a charging station in our strategy.

• 26.Another article about finding humans from camera imagery, this time combined with google maps data and includes an analysis of false positive rates.

Users

What do the users require

Our users require a way to quickly get information about a large disaster. This would mean that we must automate this information gathering on different scales. The users have the problem that they cannot get to a disaster quick enough, and when they are at the disaster, they cannot get information quick enough because of the scale. For example, when there is a very large earthquake. The emergency services have no good way to get to the disaster, they do no immediately know the scale of the disaster and they do not know which parts really require there attention. This all costs a lot of time, which can be greatly reduced. To get all this information really quick, they sometimes use drones. These are manual controlled. This means that they can only gather information at as many places as they have people available. If we can make the robots independent and automated, while communicating with each other and giving important information to the users, this process would become much faster.

Who are the stakeholders?

There are different stakeholders with different roles in this project involved. They would all take advantage of a solution we provide to their problem. The three stakeholders are Users, Society and Enterprise. We will describe per category why this particular stakeholder is involved into our problem and how our project will contribute to a solution for their problems.

Users

The biggest group of stakeholders are the users, which consists of civilians, government organizations, and private organizations or non-government organizations. These would all take advantage of the solution we provide, in particular those which are our intended end-user, i.e. the groups which will be involved during a natural disaster. We shall describe how these groups use our solution

• Government Organizations

Organizations formed by the government to combat natural disasters will take the most advantage from our solution. When a natural disaster will take place on large scale, emergency services or other organizations want to gather information as quick as possible. With our solution, this will become automated and much quicker.

• Civilians

Civilians struck by natural disasters benefit from our solution. The quicker help comes, the smaller problems arising for civilians will be. This counts for medical care, but also search and rescue and preventing loss of private property.

• Private organizations/non-government organizations

Organizations could also use our solution to work for different purposes. For example as security of property. Next to that, our solution to the described problem could be used as a good solution for similar problems as government organizations are describing.

Society

The society in a whole would benefit greatly from our solution. Our solution is relative cheap, and would be a great addition or replacement to existing solutions. Our solution would contribute to prevent loss of life, loss of property and would help organizations greatly. Next to that, since it is not a expensive solution, it would be much more cost effective than existing solutions such as the manual controlled drone.

Enterprise

The enterprise would also benefit from our solution. Firstly, the usage of drones would be far greater than before. This would mean that enterprises could cash in into our solutions.

Model

Below is a rough sketch of the UML class diagram of our software. This will not be the final version, and will be updated later on.

Drone

There is a class for the drone itself. In here, some important information about the drone is included, like position of the drone, angle of the drone etc.. It uses PIDController to stabilize the drone and to be able to set the drone to a certain angle. PID is a method to stabilize a robot.

AI

The AI is a abstract class, which can be implemented by other classes. This ensures that we can create different type of AI's. For example, one which only lets the drone go in cirkels, and another which only lets the drone draw squares. It changes its behavior depending on the context. This is important, for example when the drone has found a person. Then we want to be able to change the behavior of the drone.

Controller

The controller class has a list of AI's. It updates the AI and in the future perhaps get information from the AI in order to decide which strategie must be applied, or even update the AI on certain things.

UML Class diagram