PRE2022 3 Group1

Group members

Name	Student number	Major
Geert Touw	1579916	BAP
Luc van Burik	1549030	BAP
Victor le Fevre	1603612	BAP
Thijs Egbers	1692186	BCS
Adrian Kondanari	1624539	BW
Aron van Cauter	1582917	BBT

Project plan

Problem statement and objectives

In a 2020 study researchers examined 100 reports of a man overboard (MOB) incident using 114 parameters to create a MOB event profile. In 88 cases the casualty was deceased as a result of the MOB incident, from those 88 cases 34 were assumed dead and 54 were witnessed dead. From the witness deaths 18 died before the rescue and 31 after the rescue, in 5 cases it is unknown. The cause of death was indicated for 42 cases, the most common cause is identified as drowning (26), followed by trauma (9), cardiac arrest (4) and hypothermia (3). Based on this study the survival chance of an MOB event is slim (12%). We aim to increase the survival chance by developing a drone that can locate the victim within X minutes and provided the victim with life saving equipment such as a ring life buoy and ... .

Scenario

The 2020 study showed that 53% of the MOB events happen on cargo ships, our focus therefore lies on developing a drone for a cargo ship. We assume that the weather conditions are mild at the time of accident and during the search and rescue operation, i.e. mild waves, a mild breeze, little rain, and an average sea air temperature. We also assume that is nighttime at the time of accident and that there is a sole victim who is capable of maintaining afloat, i.e. no major/life-threating injuries.

Assume that we know a man has fallen overboard within X minutes so that we know the general search area.

Users

The end users of our product are personnel of cargo ships, shipping companies that own a fleet of cargo ships, ports, and possibly the military.

Requirements: MoSCoW method

Must

• Have sensors to detect a person within X minutes

• Have a communication system to communicate with the involved ship
• Be able to withstand an air temperature range of -3°C to 36°C.
• Be able to fly in the dark
• Be able to fly in the rain
• Be able to withstand wind forces of 4 beaufort.

Should

• Have life assist systems
• Resist (bad) weather to a degree
• A person recognition algorithm

Could

• Have communication between victim and ship

Won’t

• Have the ability to take the person to safety

Approach milestones and deliverables

TODO

Task division

Person	Week 1	Week 2	Week 3	Week 4	Week 5	Week 6	Week 7	Week 8
Luc	Literature study (4-5 articles)+Subject picking	Further Brainstorming and subject refining	Literature study (4-5 articles): Search patterns, Remote drone control > necessary equipment	Search pattern and area, TIC, (remote control)
Thijs	Literature study (4-5 articles)+Subject picking	Further Brainstorming and subject refining	Literature study(4-5 articles): Communication systems > necessary equipment	Communication systems
Geert	Literature study (4-5 articles)+Subject picking	Further Brainstorming and subject refining	Literature study(4-5 articles):  Influence of cold water, Oceanic Weather	Ethics of rescue robots, write on scenario (specifics)
Victor	Literature study (4-5 articles)+Subject picking	Further Brainstorming and subject refining	Literature study(4-5 articles): Image recognition /sensors > necessary equipment, Take care of the wiki page	TIC, other sensors and image recognition
Adrian	Literature study (4-5 articles)+Subject picking	Further Brainstorming and subject refining	Literature study(4-5 articles):  Night time deck procedures > What does the victim have on him/her?	Finalize state-of-the-art, communication system (from thursday)
Aron	Literature study (4-5 articles)+Subject picking	Further Brainstorming and subject refining	Literature study(4-5 articles):  Current tech > write a section on the current state of person detection tech in the wiki	Look into dropping equipment from the drone. (take into account wind direction and ocean currents)

State of the art: Literature Study

Search patterns

Search procedures for when a man goes overboard already exist. In the IAMSAR manual some of these procedures are already explained. I want to start discussing some of the search patterns described.

For different environments and conditions different patterns are recommended. When the location of the Man overboard is known well, Expanding Square or Sector search is recommended. if the location of the accident is not accurately known. different patterns are recommended such as sweep search.

• Expanding Square search - expanding square search can only be done by one ship at a time. the pattern is starts at the approximate location of the man over board and spirals outwards with course alterations of 90°

• Sector Search - this can be done using one vessel, or using a vessel and an aircraft. the method is used to search a spherical area. the pattern is depicted in .....

• Sweep search - this is a pattern used to search a bounded area. the ship zig zags down it.

IAMSAR also mentions other factors need to be taken into account. If a person falls into the water they will for example be moved away by currents.

Remote Drone Control

A lot of research has already gone into autonomous drone flight. Take a look at autonomous drone racing for example (https://link.springer.com/article/10.1007/s10514-021-10011-y) . A lot of consumer drones also are capable of autonomous flight, for example DJI has a waypoint system based on GPS coordinates and their drones are also capable of tracking a moving person/object. DJI's waypoint system works by loading a set of GPS coordinates onto the drone, the drone will navigate itself to the first set coordinates. From there it goes on to the next. Until it has reached the final set of coordinates. The moving object tracking is a bit more complicated. This involves image recognition.

http://acta.uni-obuda.hu/Stojcsics_56.pdf

Communication Systems

(Thijs)

Influence of cold water and oceanic weather

(Geert)

Sensors

The main sensor that will be used to locate a victim is a thermal imaging camera (TIC). The sheer size of the ocean makes it nearly impossible to use a normal camera to detect a person, it is as looking for a needle in a hay stack. Since the general search area is small relative to the complete ocean, temperatures of the water will all be relatively the same. The victim will have a much higher temperature than their surroundings, making it easy to detect them. There are a couple important requirements for the TIC. The camera needs a sufficiently high resolution so that we can clearly distinguish the victim from other warm object (think of the marine life). It also needs to have a high refresh rate. Our goal is to detect and locate a victim as soon as possible so that the survival chance is the highest, we therefore need to 'scan' the search area fast and that requires a sufficient refresh rate. For the same reason the thermal camera is required to have a large field of view (FOV). It also needs the right temperature sensitivity, and a temperature bar in the interface.

Other sensors that we might want to include are a microphone and a loudspeaker. This would allow us to make contact with the victim. However, the issue with a microphone is that the sound of the ocean is most likely so loud that it is unlikely to understand the victim, trying to conversate with the victim might also fatigue them. A loudspeaker can be used to notify the victim rescue is on the way and help them calm down.

We also aim to provide life saving equipment such as a ring life buoy, this buoy can be attached to the drone and dropped by a mechanism making use of servo motors. It is harder than it sounds to drop something near the victim from an altitude above the ocean, wind sensors should be added so that calculations can be made to drop the buoy in close proximity of the victim.

TIC

https://reprapworld.nl/elektronica/electronische-3d-merken/raspberry/raspberry-pi-camera-v2-infrarood-8mp-3280x2464-resolutie/&f=gs?utm_source=beslist&utm_medium=referral&bl3nlclid=e6745299-134d-4654-a46a-bc0da1bb118f&utm_term=BESLIe6745299-134d-4654-a46a-bc0da1bb118f

This TIC is able to take pictures with a resolution of 3280 x 2464 pixels, and videos of 1080p at 30 fps, 720p at 60 fps and 640 x 480p at 90 fps. It has an angle of view (AOV) of 67 degrees

Lets assume a human head has a width of 20 cm and length of 25 cm, the surface area would be 500 cm². If we fly the drone 500m above the water we would cover an area of 438km², however, this means each pixel represents an area of 541.6 cm² in picture mode, meaning we would not be able to recognize a person since it would not even be represented by one pixel.

Thermal image recognition

Sosnowski, T. R., Bieszczad, G., & Madura, H. (2018). Image Processing in Thermal Cameras. Studies in systems, decision and control, 35–57. https://doi.org/10.1007/978-3-319-64674-9_3

Tsai, P., Liao, C., & Yuan, S. (2022). Using Deep Learning with Thermal Imaging for Human Detection in Heavy Smoke Scenarios. Sensors, 22(14), 5351. https://doi.org/10.3390/s22145351

Night time deck Procedures

(Adrian)

Current Person in water detection tech

Currently, radars are used to detect the distance and velocity of an object by sending electromagnetic waves out and detecting the echo that results from objects. In water environments, a radome is needed to protect the radar but let the electromagnetic waves through. However, even with a radome the signal strength already falls to a third in a wet environment compared to dry. Multiple radar technologies exist with each their different advantages and disadvantages. Pulse Coherent Radar (PCR) can pulse the transmitting signal so it only uses 1% of the energy. Another aspect of this radar is coherence. This means that the signal has a consistent time and phase so the measurements can be incredibly precise. It can also seperate the amplitude, time and phase of the received signal to identify different materials. This is not necessarily needed in our use case.

The 'Sparse' service is the best service for detecting objects since it samples waves every 6cm. You don't want millimeter precise measurements in a rough environment like the ocean so these robust measurements are ideal.

Frotan, D., & Moths, J. (n.d.). Human body presence detection in water environments using pulse ... www.diva-portal.org. Retrieved March 6, 2023, from https://mau.diva-portal.org/smash/get/diva2:1668701/FULLTEXT02.pdf

Design phase

Components:

Sensors

• TIC
• Wind sensor
• Servomotors

Communication Equipment

• Microphone
• Loudspeaker

Drone

Brainstorm Phase

Possible projects

Man over board (MOB) drone (Final project)

During the MOB protocol, the most challenging part is locating the victim. This can prove to be especially difficult during stormy weather or night time. Creating a drone that is equipped with adequate sensors to locate the victim and life saving equipment would drastically increase the chances of survival for a man overboard. Another problem that our drone needs to tackle is providing appropriate care for the possibilities of drowning, hypothermia or any other injury.

User: Ship's crew, rescue teams (coast guard); Problem: MOB, Requirement: Locate and provide appropriate care for the victim.

References: https://www.ussailing.org/news/man-overboard-recovery-procedure/, https://doi.org/10.1016/j.proeng.2012.06.236

Manure Silo suffacation

Manure Silo's need to be cleaned. When this is done, people can sufficate in the toxic gasses released by the manure (even if the silo is almost empty). We want to develop a robot that alarms people when conditions become dangerous, and if this person is not able to leave the silo in time, supply clean air to them.

User: Farmers, Problem: Manure silo suffication, Requirement: Supply clean air before suffication.

Some references: https://www.ad.nl/binnenland/vader-beukt-wanhopig-in-op-silo-maar-zoon-bezwijkt~a4159109/, https://www.mestverwaarding.nl/kenniscentrum/1309/twee-gewonden-bij-ongeval-met-mestsilo-in-slootdorp

Extreme Sports Accidents

Thrillseekers are often a bit reckless when it comes to safety. We want to design a flying drone that can help people get out of sticky situations during parachute-jumping, base-jumping or even rock-climbing. The victim will be able to attach themselves to the drone using the parachute equipment or rock-climbing equipment and the drone will put them safely on the ground.

User: Extreme sporters, rescue teams, Problem: Dangerous accidents, Requirement: Can safely attach to people and put them on the ground.

Some references: https://www.nzherald.co.nz/travel/aussie-base-jumpers-two-hour-ordeal-after-parachute-gets-stuck-in-tree/HCN6DYMSSA4ZUBE3GVV2WCTNRQ/, https://www.tmz.com/2022/11/30/base-jumper-crash-cliff-dangling-parachute-death-defying-video-moab-tombstone-utah/

Literature study

To determine the state of the art surrounding our project we will do a literature study.

Disaster robotics

This article gives an overview of rescue robotics and some characteristics that may be used to classify them. The article also contains a case study of the Fukushima-Daiichi Nuclear power plant accident that gives an overview of how some robots where used. On top of that the article gives some challenges that are still present with rescue robotics.

https://link.springer.com/chapter/10.1007/978-3-319-32552-1_60

A Survey on Unmanned Surface Vehicles for Disaster Robotics: Main Challenges and Directions

This article gives an overview of the use of unmanned surface vehicles and gives some recommendations around USV's.

https://www.mdpi.com/1424-8220/19/3/702?ref=https://githubhelp.com

Underwater Research and Rescue Robot

This article is about an underwater rescue robot that gives necessary feedback in rescuing missions. This underwater robot has more computng power than the current underwater drones and reduces delay by the use of ethernet cable.

https://www.researchgate.net/publication/336628369_Underwater_Research_and_Rescue_Robot

Mechanical Construction and Propulsion Analysis of a Rescue Underwater Robot in the case of Drowning Persons

This article is about a unmanned life-saving system that recovers conscious or unconscious people. This prevents other people from getting themselves in a dangerous situation by trying to save others. This drone is not fully autonomous since it needs to be operated by humans.

https://www.mdpi.com/2076-3417/8/5/693

Design and Dynamic Performance Research of Underwater Inspection Robots

Power plants along the coastline use water as cooling water. The underwater drone presented in this paper is used to research water near power plants and clean filtering systems to optimize the efficiency of the powerplant.

https://www.hindawi.com/journals/wcmc/2022/3715514/

Semi Wireless Underwater Recue Drone with Robotic Arm

This article highlights the challenges concerning underwater rescue of people and valuable object. The biggest challenge is wireless communication due to the harsh environment. The drone is also equipped with a robotic arm to grab objects and a 4K camera with foglights to navigate properly underwater.https://www.researchgate.net/publication/363737479_Semi_Wireless_Underwater_Rescue_Drone_with_Robotic_Armhttps://www.researchgate.net/publication/363737479_Semi_Wireless_Underwater_Rescue_Drone_with_Robotic_Arm

Rescue Robots and Systems in Japan

This paper discusses the development of intelligent rescue systems using high-information and robot technology to mitigate disaster damages, particularly in Japan following the 1995 Hanshin-Awaji earthquake. The focus is on developing robots that can work in real disaster sites for search and rescue tasks. The paper provides an overview of the problem domain of earthquake disasters and search and rescue processes.

https://ieeexplore.ieee.org/abstract/document/1521744

Two multi-linked rescue robots: design, construction and field tests

This paper proposes the design and testing of two rescue robots, a cutting robot and a jack robot, for use in search and rescue missions. They can penetrate narrow gaps and hazardous locations to cut obstacles and lift heavy debris. Field tests demonstrate their mobility, cutting, and lift-up capacity, showing their potential use in rescue operations.

https://www.jstage.jst.go.jp/article/jamdsm/10/6/10_2016jamdsm0089/_pdf/-char/ja

The current state and future outlook of rescue robotics

This paper surveys the current state of robotic technologies for post-disaster scenarios, and assesses their readiness with respect to the needs of first responders and disaster recovery efforts. The survey covers ground and aerial robots, marine and amphibious systems, and human-robot control interfaces. Expert opinions from emergency response stakeholders and researchers are gathered to guide future research towards developing technologies that will make an impact in real-world disaster response and recovery.

https://doi.org/10.1002/rob.21887

Mobile Rescue Robot for Human Body Detection in Rescue Operation of Disaster

The paper proposes a mobile robot based on a wireless sensor network to detect and rescue people in emergency situations caused by disasters. The robot uses sensors and cameras to detect human presence and condition, and communicates with a network of other robots to coordinate rescue efforts. The goal is to improve the speed and efficiency of rescues in order to save more lives.https://d1wqtxts1xzle7.cloudfront.net/58969822/12_Mobile20190420-67929-tn7req-libre.pdf?1555765880=&response-content-disposition=inline%3B+filename%3DMobile_Rescue_Robot_for_Human_Body_Detec.pdf&Expires=1676230737&Signature=YQXJqYheT6M0hsHXSWDx4FbuCauvv9o9uvDR1Hl8dJL~SmI~KObXAhXbq7dDYZAMLhsydh7ipP5RBOayNkzsM~K0xP7pcXLmOKcW3-WFdt1aTyHvQWeG5hUKzhb5KLaVAj4Frfb313Yi5oyhFaHVb~ODSxbtpN73SGd3YE3UouzuexfeGSVqFyWTWi-3qMqMIQ3qfUKGiBF24QfyArHlj9mKkq8gVItdJsAS9OGBUGeBQaf~8j37WsIauoABw8cO5V73RFxhfLR~ehXXMgJegTRxzwT1tBMhE14OVMK~PkfcpYSAVkHFi3gqf~sawW4SFIut7MetNdUcKfcAwHEBHA__&Key-Pair-Id=APKAJLOHF5GGSLRBV4ZA

Mine Rescue Robot System – A Review

Underground mining has a lot of risks and it is a very difficult task for rescuers to reach trapped miners. It is therefore great to deploy a wireless robot in this situation with gas sensors and cameras, to inform rescuers about the state of the trapped miners.

https://www.sciencedirect.com/science/article/pii/S187852201500096X

Ethical concerns in rescue robotics: a scoping review

We also have to take the ethics of rescue robots into account. There are seven core ethical themes: fairness and discrimination; false or excessive expectations; labor replacement; privacy; responsibility; safety; trust

https://link.springer.com/article/10.1007/s10676-021-09603-0

Rescue robots for mudslides: A descriptive study of the 2005 La Conchita mudslide response

Robots assisted the rescuers who responded to the 2005 mudslide in La Conchita. The robots were waterproof and could thus be deployed in wet conditions, but they failed to navigate through the rubble, vegetation and soil. The paper thus suggests that rescue robots should be trained in a variety of environments, and advises manufacturers to be more conservative with their performance claims.

https://onlinelibrary.wiley.com/doi/abs/10.1002/rob.20207

Emergency response to the nuclear accident at the Fukushima Daiichi Nuclear Power Plants using mobile rescue robots

The 2011 earthquake and tsunami in Japan resulted in a meltdown of the Fukushima nuclear power plant. Due to the radiation levels, robots were deployed because it was too dangerous for humans. First various issues needed to be resolved, like the ability of the robot’s electrical components to withstand radiation. The ability to navigate and communicate was tested at a different nuclear powerplant similar to Fukushima.

https://onlinelibrary.wiley.com/doi/full/10.1002/rob.21439

A Coalition Formation Algorithm for Multi-Robot Task Allocation in Large-Scale Natural Disasters

Robots are more reliable then humans in a lot of cases. This paper discusses a bit of prior research concerning older algorithms and looks into a new algorithm considering multi-robot task allocation is rescue situations. These algorithms should take a lot into account, like sensors needed for problems. They compare their algorithm with older ones in multiple cases like different problem sizes.

https://www.researchgate.net/publication/316283106_A_Coalition_Formation_Algorithm_for_Multi-Robot_Task_Allocation_in_Large-Scale_Natural_Disasters

Appendix

Logbook

Week	Name	Work done & hours spent	Total hours
1	Geert Touw	Group setup etc (2h)
	Luc van Burik	Group setup etc (2h), Subject brainstorm (2 h)	4h
	Victor le Fevre	Group setup etc (2h)
	Thijs Egbers	Group setup etc (2h)
	Adrian Kondanari	Group setup etc (2h)
	Aron van Cauter	Group setup etc (2h)
2	Geert Touw	Meeting 1 (1h), Meeting 2(1h)
	Luc van Burik	Meeting 1 (1h), Meeting 2(1h), Subject brainstorm (2h)	4h
	Victor le Fevre	Meeting 1 (1h), Meeting 2(1h)
	Thijs Egbers	Meeting 1 (1h), Meeting 2(1h), Subject brainstorm (2h)	4h
	Adrian Kondanari	Meeting 1 (1h)
	Aron van Cauter	Meeting 1 (1h)
3	Geert Touw
	Luc van Burik	Meeting 2 (1h), IAMSAR reading (3.5h), Drone navigation research (1.5h)	6h
	Victor le Fevre	Meeting 1 (1h), meeting 2 (1h) Made the wiki look a bit more coherent (1h) Wrote the problem statement and objective, users, scenario and added points to the MoSCoW (1.5h) Read Sensor and Image Recognition papers (1h)	5.5h
	Thijs Egbers
	Adrian Kondanari
	Aron van Cauter
4	Geert Touw	Meeting 1 (1h)
	Luc van Burik	Meeting 1 (1h)
	Victor le Fevre	Meeting 1 (1h)
	Thijs Egbers	Meeting 1 (1h)
	Adrian Kondanari	Meeting 1 (1h)
	Aron van Cauter
5	Geert Touw
	Luc van Burik
	Victor le Fevre
	Thijs Egbers
	Adrian Kondanari
	Aron van Cauter
6	Geert Touw
	Luc van Burik
	Victor le Fevre
	Thijs Egbers
	Adrian Kondanari
	Aron van Cauter
7	Geert Touw
	Luc van Burik
	Victor le Fevre
	Thijs Egbers
	Adrian Kondanari
	Aron van Cauter
8	Geert Touw
	Luc van Burik
	Victor le Fevre
	Thijs Egbers
	Adrian Kondanari
	Aron van Cauter

Things things we might need later (to be deleted)

Weather: Some waves, a breeze but still calm enough for a drone to fly, little rain, nighttime, water is 3 degrees Celsius -> 15-30 minutes until exhaustion and unconsciousness, 30-90 minutes expected survival time.

Ship: Container ship, on the Atlantic ocean, speed: 25 knots (~46 km/h, had to take a detour, were behind schedule, faster than average)

Reason off fall: Dark outside, slippery because some rain, person is alone, person drifts away without major injuries