PRE2018 1 Group1

Preface

Group members

Name	Study	Student ID
Cornet, N.	Industrial Design	1007926
Horssen, C.	Software Science	0885378
Mouw, F.A.	Applied Physics	1005735
Stokbroekx, D.L.M.	Mechanical Engineering	1010326
Buijvoets D.C.J.T	Mechanical & Electrical Engineering	0902148

Initial robotic concepts

After discussing several subjects from different fields in society, we came up with the following list of robotic concepts which have potential to solve problems faced by certain users.

• Fire fighter drone which can be used to aid fire fighters
• Cleaning drone for difficult to reach spots in buildings
• Pavement cleaning drone which can remove dirt from tiles
• Avalanche rescue drone that helps rescuing teams search for victims in an avalanche using already available beacons
• Weeding robot which can differentiate between wanted and unwanted plants in a garden and remove the weeds
• Referee robot using image processing to determine the state of a match in e.g. soccer or tennis
• Medical nanobots for drug delivery
• Fruit harvest robots

Chosen concept: Flying Fire Fighter

Eventually we agreed upon the concept of a fire fighter drone. Fire fighting is one of the most dangerous jobs and every year people are still being killed by fires. Extra preparation and information on the fire site can make the difference between life and death. That is where we thought a drone could be of help.

Problem statement

How can drones be used for fully autonomous fire fighting in smart cities. In this project the focus will be on the fire extinguishing part of fire fighting.

Objectives

• The fire fighting robot needs to give victims information about what they can do to improve their chances of survival without injury
• It needs to be able to communicate the current state of the fire back to the fire department to inform and prepare firefighters
• The drone has to be able to enter and fly inside a building autonomously
• It has to have the ability to extinguish small fires

During this project, our focus will especially be on the last objective.

USE analysis

Relevant users and their requirements

• Fire fighters will require a fast response time to the fire and an accurate analysis of the fire site
• Victims require information on how to reduce harm to themselves and others and a safe rescue

Because no contact could be found with any victims, and that there was no chance to visit the fire department yet, the following requirements are based upon the usage of common sense and the information found on victim support sites. These are temporary and will be further validated once contact with experts has been made.

Two options

• Firedepartment owns the drones
• Drones are part of smart home package (in that case, building owner becomes main stakeholders)

Main users

• Fire fighters/Fire Department

The fire department uses the drone in order to put out fires more easily, safer and faster. Because the drone can be at the site of the fire faster than the fire fighters, the fire will get less time to spread out and victims have a bigger chance of survival. Furthermore if a part of the fire is already extinguished, the firefighters themselves are less at risk to sustain serious or even lethal injuries.

Requirements:

• Fire localization

The drone needs to be able to search for and recognize an ongoing fire

• Autonomous flight

The drone need to be able to fly to the site of the fire autonomously

• Long durability

The drone needs to be fire resistant and to be able to withstand at least XXX missions.

• Long battery life

The drone needs to be able to fly around actively for at least XXX minutes

• Victim recognition

The drone needs to be able to recognize humans/victims close to the fire

• Able to extinguish small fires

The drone needs to be able to extinguish small fires (max. AFMETINGEN)

• Situation analysis

The drone needs to be able to analyse the situation, the damage and if the call was a false alarm or not

• Fire victims

Being amids a fire, is a very stressful and traumatic experience.Victims during a fire can either be conscious or unconscious. When conscious, the victim needs to be able to be reassured that help is on its way and to stay as calm as possible. When unconscious it is important that the victim attains as little (further) damage or injuries as possible.

Requirements:

• Make victim feel safe

The drone needs to try to keep the victim as calm as possible, by assuring that help is on its way and to explain every action it is taken. The victim should not feel threatened by the drone

• Give instructions

After analysing a situation, the drone needs to, if possible, give instructions to the victim of possible actions to improve chance of survival of the victim

• Cannot (lethally) harm victim in any way

The drone is not allowed to harm the victim in any way, or do something that brings lethal injury to the victim.

Other stakeholders

Building owners

Building owners have

Requirements:

Building should sustain as less damage as possible

Relevance to society

• People living in range of the fire station using this drone will be subject to better rescuing in case of a fire
• If the drone is being used to aid at fighting a fire, the people living in the viscinity of that fire will have a smaller chance that the fire will affect them

Relevance to enterprise

• The government
• Fire department

Project setup

Before actually starting the project, a setup is made on how it will be executed.

Approach

• Decide functionality of the drone
• Do research on different subjects concerning the functionality of the drone
• Design drone and functions
• Make prototype
• Test prototype

Milestones

For the duration of this course, the following milestones are selected:

• Week 1: Every member will take the time to do research on robots and their interests, in order to broaden one's horizon on the possible subjects. Afterwards a subject for the project will be chosen.
• Week 2: Literature study and further research will be completed
• Week 3: USE analysis is finished
• Week 4: Design for the first prototype will be finished
• Week 6: First prototype will be finished
• Week 8: Final Design, final prototype and all other deliverables will be finished

Deliverables

The following deliverables will be created during this course:

• Thorough research and literature study
• Design research process and report
• A functional drone design
• (several) Prototypes
• Ethical evaluation
• A wiki page on this domain

Planning: Who's doing what

Week	All	Natanya	Chiel	Fabian	Daan	Undecided
1	Finding suitable projects + finding articles on the chosen subject	Basic user requirements			Formulating Problem statement
2		Further elaborating user requirements and USE analysis + making persona’s of the users	State of the art research	State of the art research	State of the art research	Further elaborating user requirements and USE analysis + start making designs for the robot
3	Checking wiki and correcting formulation if necessary					Continuing on design for the robot + finishing USE analysis
4			Starting on control software for the robot	Starting constructing of the prototype	Starting constructing of the prototype	Finishing robot design
5	Checking wiki and correcting formulation if necessary			Construction of the prototype	Construction of the prototype
6	Testing and evaluating prototype + find and analyse flaws in the design + fixing the design flaws			First prototype finished	First prototype finished
7	Find and analyse flaws in the design + fixing the design flaws + checking wiki and correcting formulation if necessary	Preparing presentation	Preparing presentation
8	Final design and robot finished + Presentation of the result

State of the art: Literature study

One of the most important facets of this project is to come to an understanding of the current state of technologic advancement that is relevant to the drone and its functionalities. Therefore, literature on different relevant catagories is studied to understand the state of the art. Here follows a list of the scientific research that was studied for this project, divided into the different relevant catagories.

Drones in firefighting

Drones and robots are already extensively used in firefighting but there are few examples of actual autonomous fire extinguishing drones. Most drones in firefighting are used to support humans by providing vital information on the emergency.

Drones provide information on how fires are spreading and developing to firefighters to increases the efficiency of the process by sending units to right places. The information gathered by these drones can also be further analyzed for a better understanding of how fire spreads for optimization of extinguishing techniques. One of the largest fire departments in the world the L.A.F.D. (Los Angeles Fire Department) is currently researching and using these kind of drones [1]. The drones have been used for the first time during the massive wildfires in December 2017 and have been a great help in the firefighting process. These drones do still require a human pilot and a human to analyze the information.

Drones also help emergency services with other non-fire related disasters. For example, during floods or earthquakes these drones can be in the disaster area much faster to collect information on the situation for the emergency services.

A different application for drones in firefighting is the localization of victims in fires that would otherwise never be found. Currently one of the best drones for this undertaking is the Firestorm UAV [2]. This drone can find victims inside burning buildings using a thermal camera also this drone can detect toxic gasses and inform firefighters on the situation in a building. Using bright LED lights, the drone can lead a localized victim along the safest path outside of the building.

The emergency services also use the drones to make emergency deliveries to certain disaster areas. The payloads of the drones can be extremely diverse, from AED machines, medical- and food supplies. One of the drones currently in development for this purpose by the company ZIPLINE [3]. The drone that is currently being tested in the USA and already operational in Rwanda for blood deliveries to rural hard to get areas drops the payload mid-air before it flies back to base to be resupplied and launched again.

Another way in which drones are used by fire departments is to make pre-fire plans for high risk or vital buildings. These drones can map escape routes and localize water supplies and potential problems for these buildings.

There are some drones that can also extinguish fire. But these drones are not autonomous and require a human pilot or firetruck for water supply and further support. An example of drones used for actual fire extinguishing are the drones produced by the company AERONES [4]. Although these drones are capable of combatting fires they cannot operate without the presence of a fire truck supporting the drone with water and electricity. This drone can reach heights of 300 meters which is much higher than the height a traditional fire truck can reach. However due to the hoses connected to the drone it can only be used for combating the fire from the exterior. This drone can be useful for fires in high-rise buildings or at other great heights that were traditionally hard to reach.

[1] https://uavcoach.com/tipping-points/

[2] https://designmind.frogdesign.com/2014/03/drones-will-save-life/

[3] https://www.theverge.com/2018/4/13/17206398/zipline-drones-delivery-blood-emergency-medical-supplies-startup-rwanda-tanzania

[4] https://www.aerones.com/eng/drones/firefighting_drone/

General Drone Information

• Remington, Raquel, et al. "Multi-Purpose Aerial Drone for Bridge Inspection and Fire Extinguishing." (Unpublished Thesis). Florida International University. Retrieved April 10 (2014): 2016. (Fabian)
• Suresh, Jayanth. "Fire-fighting robot." Computational Intelligence in Data Science (ICCIDS), 2017 International Conference on. IEEE, 2017.(Fabian)
• Design of a portable robot/device that is able to gather environmental information about the fire and guide victims for evacuation: Kim, Y.-D., Kim, Y.-G., Lee, S.-H., Kang, J.-H., An, J. “Portable fire evacuation guide robot system” (2009) IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2009, art. no. 5353970, pp. 2789-2794. (Daan)

Autonomous drone navigation

• General controller for safer autonomous navigation: Pestana, J., Mellado-Bataller, I., Fu, C., Sanchez-Lopez, J.L., Mondragon, I.F., Campoy, P. “A general purpose configurable navigation controller for micro aerial multirotor vehicles” (2013) International Conference on Unmanned Aircraft Systems, ICUAS 2013 - Conference Proceedings, art. no. 6564733, pp. 557-564. (Daan)
• Complete navigation system using 3D laser scanner for omnidirectional environment perception, local and allocentric maps for positioning and a multi-layered approach for trajectory planning (global mission trajectory and local obstacle avoidance): Nieuwenhuisen, M., Droeschel, D., Beul, M., Behnke, S. “Autonomous Navigation for Micro Aerial Vehicles in Complex GNSS-denied Environments” (2016) Journal of Intelligent and Robotic Systems: Theory and Applications, 84(1-4), pp. 199-216. (Daan)
• Obstacle avoidance and field planning using monocular sensory input: Mac, T.T., Copot, C., Hernandez, A., De Keyser, R. “Improved potential field method for unknown obstacle avoidance using UAV in indoor environment” (2016) SAMI 2016 - IEEE 14th International Symposium on Applied Machine Intelligence and Informatics - Proceedings, art. no. 7423032, pp. 345-350. (Daan)

Autonomous victim detection

• Detecting injured humans on images taken from aerial vehicles: ANDRILUKA, Mykhaylo, et al. Vision based victim detection from unmanned aerial vehicles. In: Intelligent Robots and Systems (IROS), 2010 IEEE/RSJ International Conference on. IEEE, 2010. p. 1740-1747. (Chiel)
• Building maps and marking victims on those maps using hyperspectral imaging: TRIERSCHEID, Marina, et al. Hyperspectral imaging or victim detection with rescue robots. In: Safety, Security and Rescue Robotics, 2008. SSRR 2008. IEEE International Workshop on. IEEE, 2008. p. 7-12. (Chiel)
• Victim detection using an adapted Viola-Jones algorithm: DE CUBBER, Geert; MARTON, Gabor. Human victim detection. In: Third International Workshop on Robotics for risky interventions and Environmental Surveillance-Maintenance, RISE. 2009. (Chiel)
• Using ad-hoc network with base station (firetruck or fire department?): SUGIYAMA, Hisayoshi; TSUJIOKA, Tetsuo; MURATA, Masashi. Victim Detection System for Urban Search and Rescue Based on Active Network Operation. In: HIS. 2003. p. 1104-1113. (Chiel)
• False positive reduction on victim detection from colored images: KLEINER, Alexander; KUMMERLE, Rainer. Genetic MRF model optimization for real-time victim detection in search and rescue. In: Intelligent Robots and Systems, 2007. IROS 2007. IEEE/RSJ International Conference on. IEEE, 2007. p. 3025-3030. (Chiel)
• Detecting victims using pseudo-noise radars, whose signals scatter from body motions of victims: SACHS, Jürgen, et al. Trapped victim detection by pseudo-noise radar. In: Proceedings of the 1st International Conference on Wireless Technologies for Humanitarian Relief. ACM, 2011. p. 265-272. (Chiel)

Fire detection

• Detecting fire from colored images, distinguishing fire and smoke: CHEN, Thou-Ho; WU, Ping-Hsueh; CHIOU, Yung-Chuen. An early fire-detection method based on image processing. In: Image Processing, 2004. ICIP'04. 2004 International Conference on. IEEE, 2004. p. 1707-1710. (Chiel)
• Detecting fire using space-time fluctuations on colored images: YAMAGISHI, Hideaki; YAMAGUCHI, JUNICHI. Fire flame detection algorithm using a color camera. In: Micromechatronics and Human Science, 1999. MHS'99. Proceedings of 1999 International Symposium on. IEEE, 1999. p. 255-260. (Chiel)
• Detecting fire using Gaussian distributions: CELIK, Turgay, et al. Fire detection using statistical color model in video sequences. Journal of Visual Communication and Image Representation, 2007, 18.2: 176-185. (Chiel)
• Fire detection in tunnels using cameras and infrared: NODA, S.; UEDA, K. Fire detection in tunnels using an image processing method. In: Vehicle Navigation and Information Systems Conference, 1994. Proceedings., 1994. IEEE, 1994. p. 57-62. (Chiel)

Fire suppression

• Experiments of different solid particulate aerosol suppressants in the form of a solid, gel or powder: Kibert, C.J., Dierdorf, D. “Solid particulate aerosol fire suppressants” (1994) Fire Technology, 30 (4), pp. 387-399. (Daan)
• Testing the effectiveness of using nanocomposites as additive to conventional powder suppressants: Ni, X., Kuang, K., Wang, X., Liao, G. “A New Type of BTP/Zeolites Nanocomposites as Mixed-phase Fire Suppressant: Preparation, Characterization, and Extinguishing Mechanism Discussion” (2010) Journal of Fire Sciences, 28 (1), pp. 5-25. (Daan)
• Experiments with a portable mist extinguisher for different types of fires: Liu, Z., Kim, A.K., Carpenter, D. “A study of portable water mist fire extinguishers used for extinguishment of multiple fire types” (2007) Fire Safety Journal, 42 (1), pp. 25-42. (Daan)

Fire resistant materials

• Lyon, Richard E., et al. "Fire‐resistant aluminosilicate composites." Fire and materials 21.2 (1997): 67-73. (Fabian)
• Myeong, W. C., Kwang Yik Jung, and Hyun Myung. "Development of FAROS (fire-proof drone) using an aramid fiber armor and air buffer layer." Ubiquitous Robots and Ambient Intelligence (URAI), 2017 14th International Conference on. IEEE, 2017. (Fabian)
• Myeong, Wancheol, Kwang Yik Jung, and Hyun Myung. "Development of a fire-proof aerial robot system for fire disaster." World Congress on Advances in Nano, Bio, Robotics and Energy (ANBRE). IASEM Conferences, 2017.(Fabian)
• Abbott, N. J., M. M. Schoppee, and J. Skelton. Heat Resistant and Nonflammable Materials. FABRIC RESEARCH LABS INC DEDHAM MA, 1976. (Fabian)
• Luo, Qiu-Sheng, Shi-Feng Li, and Hui-Ping Pei. "Progress in titanium fire resistant technology for aero-engine." Journal of Aerospace Power 27.12 (2012): 2763-2768.(Fabian)

Not catagorized yet

• (Up from the Rubble: Lessons Learned about HRI from Search and Rescue)

Robin R. Murphy & Jennifer L. Burke

• (Improved Interfaces for Human-Robot Interaction in Urban Search and Rescue)

Michael Baker, Robert Casey, Brenden Keyes, and Holly A. Yanco

• (Exploring 3D Gesture Metaphors for Interaction with Unmanned Aerial Vehicles)

Kevin P. Pfeil, Seng Lee Koh, Joseph J. LaViola Jr.

• (Analysis of Human-Robot Interaction for Urban Search and Rescue)

Holly A. Yanco, Michael Baker, Robert Casey, Brenden Keyes, Philip Thoren, Jill L. Drury, Douglas Few, Curtis Nielsen, David Bruemmer

• (Drone Near Me: Exploring Touch-Based Human-Drone Interaction)

PARASTOO ABTAHI, DAVID Y. ZHAO, JANE L. E, and JAMES A. LANDAY

• (Evaluation of Human-Robot Interaction Awareness in Search and Rescue)

Jean Scholtz, Jeff Young, Jill L. Drury, Holly A.Yanco

• (Human-Drone-Interaction: A Case Study to Investigate the Relation Between Autonomy and User Experience)

Patrick Christ, Axel Hösl, Florian Lachner, Klaus Dieopold

• (Emotion Encoding in Human-Drone Interaction)

Jessica R. Cauchard*, Kevin Y. Zhai, Marco Spadafora, James A. Landay

• (Human-Robot Interaction in Rescue Robotics)

Robin R. Murphy

• (Human-Robot Teaming for Search and Rescue)

Illah R. Nourbakhsh, Katia Sycara, Mary Koes, and Mark Yong, Michael Lewis, Steve Burion

• (Natural User Interfaces for Human-Drone Multi-Modal Interaction)

Ramón A. Suárez Fernández, Jose Luis Sanchez-Lopez, Carlos Sampedro, Hriday Bavle, Martin Molina, and Pascual Campoy (Drone & Wo: Cultural Influences on Human-Drone Interaction Techniques) Jane L. E, Ilene L. E, James A. Landay, Jessica R. Cauchard

• (Drone & Me: An Exploration Into Natural Human-Drone Interaction)

Jessica R. Cauchard, Jane L. E, Kevin Y. Zhai, James A. Landay

Rotterdam fire department

A conversation was held with the director of the rotterdam fire department, which actually has a fire fighting drone in operation at the moment. This drone is autonomously navigated by another (controlled) vehicle which makes a 3D map of the inside of a building. We will have the opportunity to talk to the project leader of this drone about some points of improvement.