PRE2022 3 Group2
Group members
| Name | Student Number | Study |
|---|---|---|
| Clinton Emok | 1415115 | BCS |
| Richard Farla | 1420380 | BCS |
| Yash Israni | 1415883 | BCS |
| Tessa de Jong | 1498312 | BPT |
| Kaj Scholer | 1567942 | BME |
| Pepijn Tennebroek | 1470221 | BPT |
Abstract
The use of robotics in disaster response operations has gained significant attention in recent years. In particular, low-cost vine robots have been proposed as a promising technology for search and rescue missions in urban areas affected by earthquakes. Vine robots are small, flexible and lightweight robots that can navigate through tight spaces and confined areas, making them ideal for searching collapsed buildings for survivors. They are also relatively cheaper and easier to assemble than other current search and rescue robots. However, vine robots have not been implemented into real-life search and rescue missions due to various complications. These complications have been evaluated and some solutions have been provided with regards to the sensors being used, as well as the localization and path planning of the vine robot. This paper only acts as an aid to support future implementation of vine robots into search and rescue missions.
Introduction and project goals
Earthquakes are one of the most devastating natural disasters that can occur in urban areas, leading to significant damage to infrastructure, loss of life and displacement of communities. In recent years, search and rescue operations have become increasingly important in the aftermath of earthquakes. These operations aim to locate victims trapped under collapsed buildings and provide them with the necessary medical care and assistance. However, these operations can be challenging due to the complexity and dangers associated with navigating through the rubble of damaged buildings.
To address these challenges, low-cost vine robots have emerged as a promising technology for search and rescue operations in urban areas affected by earthquakes. Vine robot are small lightweight and flexible robots that can crawl slither or climb through tight spaces and confined areas making them ideal for searching collapse buildings for survivors. these robots can be equipped with camera sensors and other devices that can aid in localization of victims.
The usage of low-cost find robots in earthquake response after it has the potential to significantly improve the efficiency and effectiveness of search and rescue operations, as well as reduce the risk to human rescuers. this paper provides an overview of the usage of low-cost van robots in the localization of victims of earthquakes that lead to infrastructure damage specifically in urban areas.
This paper presents a comprehensive review of the literature on the usage of low-cost vine robot in earthquake response efforts. The paper provides an overview of the design and capabilities of vine robots, highlighting their potential advantages over traditional search and rescue methods. This paper also discusses the challenges associated with the usage of vine robots, such as limited battery life, difficulties in controlling the robot in complex environments and the need for specialized training for operators.
To support the analysis, this paper draws on case studies of the deployment of vine robot in earthquake disasters, including the 2010 Haiti earthquake, the 2017 Mexico City earthquake and the 2019 Albania earthquake. The case studies serve to illustrate the effectiveness of vine robots in localizing victims as well as the challenges rescue teams face by utilizing this technology.
The research questions addressed in this paper include:
- What are the design and capabilities of low-cost van robots and how do they differ from 4-0 search and rescue methods?
- What are the potential advantages and challenges associated with the usage of low-cost van robots in earthquake response efforts?
- How have vine robots been deployed in recent earthquake disasters and what has their impact on search and rescue operations been?
- What are the future prospects of low-cost vine robots in earthquake response efforts?
Extra Content for Introduction (to be adjusted...)
“Two large earthquakes struck the southeastern region of Turkey near the border with Syria on Monday, killing thousands and toppling residential buildings across the region.” (AJLabs, 2023) The earthquakes were both above 7.5 on the Richter scale, which caused buildings to be displaced from foundations with people still in them. Some people survived the fall when a building collapsed, but were trapped in all of the rubble.
After earthquakes of high magnitude, it is necessary to rescue survivors from destroyed buildings as fast as possible. Namely, the chances of finding people alive in rubble fade with each passing day. However, it can be hard for human rescuers and rescue dogs to reach these areas due to the dangers of collapsing buildings. Therefore, the usage of robotics can be introduced in these rescue operations. In this report, it is investigated how the usage of vine robotics could improve localizing alive people after earthquakes of high magnitude. Furthermore, it is looked into how the vine robot can prolong survivors’ lives. This would hopefully increase the number of people that are saved after such a natural disaster. In order to do this, literature research is conducted and a simulation and prototype will be created.
For this project, we will focus on urban search and rescue. This means as stated in Wikipedia: “a type of technical rescue operation that involves the location, extrication, and initial medical stabilization of victims trapped in an urban area, namely structural collapse due to natural disasters, war, terrorism or accidents, mines and collapsed trenches.” Our scope will be the urban area after an earthquake, so we will deal with the collapsed buildings and structures that arise from them. In the scenario, where our vine robot will operate, will not be any fire or water which the robot has to take into account
Project planning and deliverables
| Week | Milestones |
|---|---|
| Week 1 | Topic, problem identification, planning, state-of-the-art literature research |
| Week 2 | Further literature study, user analysis, MoSCoW, CAD modelling, research for simulation possibility, research/order electronics |
| Week 3 | Further literature study, complete CAD modelling, start simulation, research localization methods, research sensors used |
| Week 4 | Work on prototype, work on simulation |
| Week 5 | Work on prototype, finalize simulation |
| Week 6 | Finalize prototype, gather results from testing |
| Week 7 | Evaluate results and conclusion |
| Week 8 | Complete wiki and presentation |
Who is doing what?
| Names | Tasks |
|---|---|
| Clinton Emok | Sensors and Localization |
| Richard Farla | Simulation |
| Yash Israni | Simulation |
| Tessa de Jong | Literature Research |
| Kaj Scholer | Sensors and Localization |
| Pepijn Tennebroek | Literature Research |
State-of-the-art literature
Existing rescue robots
The Current State and Future Outlook of Rescue Robotics
There exist various state-of-the-art rescue robots with promising future outlooks. However, achieving full autonomy in real-world rescue scenarios is presently challenging to implement. In fact, there is a marked inclination towards semi-autonomous behaviors, rather than complete manual control (Delmerico et al., 2019).
Robotically negotiating stairs
The technique for traversing stairs involves obtaining image data depicting a robot navigating through a stair-filled environment using either one or two legs. This method is highly comparable to crossing terrain that is cluttered with debris and fallen objects (Boston Dynamics, Inc., 2019).
Design of four-arm four-crawler disaster response robot OCTOPUS
The OCTOPUS robot, presented in this paper, boasts four arms and four crawlers, providing exceptional mobility and flexibility. Its arms are engineered with multiple degrees of freedom, enabling the robot to execute intricate tasks such as lifting heavy objects and opening doors. Furthermore, the crawlers are designed to offer stability and traction, which allow the robot to move seamlessly on irregular and slippery surfaces (Kamezaki et al., 2016).
Technology in rescue robots
Specially Designed Multi-Functional Search And Rescue Robot
In this paper, a sensor-based multi-functional search and rescue robot system for use in emergency situations is designed. They provide an insight on the various sensors that are included in this robot system, which are the following: servo motor, USB camera, DC motor, motor driver module, stepper motor, Darlington transistor arrays, ultrasonic sensor, Raspberry Pi 3 model B+ and Arduino mega. This robot system can search humans in ruined areas and send the collected data to a web server, which is then able to video steam in real-time (Nosirov et al., 2020).
Life Signs Detector Using a Drone in Disaster Zones
A new computer vision system has been developed to detect vital signs in hazardous zones using drones. The outcomes of the study indicate that the system can detect breathing patterns from an aerial platform with a high level of accuracy. The system effectively differentiates between humans and mannequins in daylight, with the aid of a human operator and a robust PC equipped with MATLAB (Al-Naji et al., 2019).
Search and Rescue System for Alive Human Detection by Semi-Autonomous Mobile Rescue Robot
This study introduces a cheap robot designed for detecting humans in perilous rescue missions. The paper presents several system block diagrams and flowcharts to demonstrate the robot's operational capabilities. The robot incorporates a PIR sensor and IP camera to detect human presence through their infrared radiation. These sensors are readily available and cost-effective compared to other urban search and rescue robots (Uddin & Islam 2016).
Legged Robots That Balance
This study has significant implications for theories on human motor control and lays a fundamental groundwork for legged locomotion, which is one of the least explored areas of robotics. The study addresses the potential of constructing functional legged robots that can run and maintain balance (Raibert, 2000).
An Overview of Legged Robots
The present paper highlights the progress and the current state-of-the-art in the field of legged locomotion systems. The study examines various possibilities for mobile robots, including artificial legged locomotion systems, and discusses their benefits and drawbacks. It explores the advantages and limitations of such systems, providing insights into their potential for future advancements (Tenreiro Machado & Silva, 2006).
Designing, developing, and deploying systems to support human–robot teams in disaster response
The focus of this paper is on the creation, construction, and implementation of systems that aid in human-robot teams during disaster response. The paper has resulted in significant advancements in robot mapping, robot autonomy for operating in challenging terrain, collaborative planning, and human-robot interaction. The presentation of information considers the fact that these contexts can be stressful, with individuals working under varying levels of cognitive load (Kruijff et al., 2014).
Perception-Driven Obstacle-Aided Locomotion for Snake Robots: The State of the Art, Challenges and Possibilities †
Snake robots have the potential to be equipped with sensors and tools to transport materials to areas that are hazardous or inaccessible to other robots and humans. Their flexible and slender design allows them to navigate through narrow and confined spaces, making them ideal for performing tasks in environments such as collapsed buildings or underground tunnels. By incorporating sensors and specialized equipment, snake robots can operate in a variety of hazardous environments, including those with high levels of radiation or toxic chemicals, without putting human workers at risk. They "expanded the description for increasing the level of autonomy within three main robot technology areas: guidance, navigation, and control" (Sanfillipo et al., 2017).
Robotic Urban Search and Rescue: A Survey from the Control Perspective
The current paper presents an extensive review of advancements in robotic control for urban search and rescue (USAR) settings, which is an exciting and challenging field of research. The paper covers the development of low-level controllers to facilitate rescue robot autonomy, task sharing between the operator and robot for multiple tasks, and high-level control schemes designed for multi-robot rescue teams. These innovations aim to improve the functionality, reliability, and effectiveness of rescue robots in disaster response scenarios, which can help save lives and minimize damage (Liu & Nejat, 2013).
Robots Gear Up for Disaster Response
Although brilliant robotic technology exists, there is a need to integrate it into complete, robust systems. Furthermore, there is a need to develop sensors and other components that are smaller, stronger, and more affordable (Anthes 2010).
Active scope camera for urban search and rescue
The focus of this paper is the design and implementation of an Active Scope Camera (ASC) for urban search and rescue (USAR) operations. The ASC is a small, lightweight device that can be deployed to explore confined spaces and provide visual information to rescuers (Hatazaki et al., 2007).
The design of telescopic universal joint for earthquake rescue robot
This paper describes a transmission system that includes a telescopic universal joint, which is used in a snake-like search and rescue robot. The paper emphasizes the importance of designing flexible and adaptable robotic systems that can be used effectively in rescue operations. The telescopic universal joint is highlighted as a promising solution to enhance the capabilities of rescue robots. The paper provides details on the design and construction of the joint, which allows the robot to navigate through narrow spaces and tight corners while maintaining its structural integrity. By improving the flexibility and mobility of rescue robots, such as with the telescopic universal joint, search and rescue operations can become more effective and efficient, potentially saving lives in critical situations (Zhao et al., 2016).
Multimodality robotic systems: Integrated combined legged-aerial mobility for subterranean search-and-rescue
This article discusses a Boston Dynamics Spot robot that is enhanced with a UAV carrier platform, and an autonomy sensor payload (Lindqvist et al., 2022). The paper demonstrates how to integrate hardware and software with each other and with the architecture of the robot.
The EU-ICARUS project: Developing assistive robotic tools for search and rescue operations
This paper describes the ICARUS tools (De Cubber et al., 2013). One of these tools is a small lightweight camera system that should be able to detect human survivors. According to this article, an infrared sensor with high sensitivity in the mid-IR wavelength range would be the most adequate detection instrument.
This paper mentions that technological tools are no good for USAR teams if they do not know how to make use of them. Therefore, training and support infrastructure is required. In order to do this, e-training and trainers-simulators can be used.
Detecting Earthquake Victims Through Walls
This article is about a new radar system that is able to detect human breathing and movement through walls (Hampson, 2022). The system uses Doppler radar to measure small changes in electromagnetic waves caused by body movements and breathing. This could be helpful in USAR operations. However, the article also mentions concerns about privacy and researchers emphasize the need for ethical considerations.
Land-Mobile Robots for Rescue and Search: A Technological and Systematic Review
This article discusses sensors that are used to obtain data on the robot’s environment based on 26 papers on rescue and search robots (Huamanchahua et al., 2022). For remote teleoperation cameras are most often used (in 96% of the cases). For identifying victims, 3D mapping and/or image processing is most often used (96%). In around 50% of the papers, microphones are used for human detection. Furthermore, CO2 sensors are used in 73% of the papers. Lastly, in 30% of the cases, temperature sensors are used to measure the victim’s body temperature.
Review Paper on Search and Rescue Robot for Victims of Earthquake and Natural Calamities
This article reviews search and rescue robots designed to locate and assist survivors of natural calamities (Bangalkar & Kharad, 2015). It concludes that these robots have the potential to save lives after natural disasters, and it mentions the need for continued research and development in this area.
Rescue robots in action
Disaster Robotics
This book offers a comprehensive overview of rescue robotics within the broader context of emergency informatics. It provides a chronological summary of the documented deployments of robots in response to disasters and analyzes them formally. The book serves as a definitive guide to the theory and practice of previous disaster robotics (Murphy, 2017).
Application of robot technologies to the disaster sites
For the first time during the Great East Japan Earthquake disaster, Japanese rescue robots were utilized in actual disaster sites. Their tele-operation function and ability to move on debris were essential due to the radioactivity and debris present (Osumi, 2014).
Utilization of Robot Systems in Disaster Sites of the Great Eastern Japan Earthquake
After deploying robots in recovery operations, three key lessons were learned. Firstly, rescue robots are valuable not only for response but also for economic and victim recovery. Secondly, disaster robots need to be optimized to suit the specific missions and stakeholder requirements. Lastly, human-robot interaction continues to pose a challenge (Matsuno et al., 2014).
Emergency response by robots to Fukushima-Daiichi accident: summary and lessons learned
Numerous lessons were learned from the emergency response of robots to the accident, with a focus on the organization and operation scheme as well as systemization (Kawatsuma et al., 2013).
Use of active scope camera in the Kumamoto Earthquake to investigate collapsed houses
The paper discusses the use of the Active Scope Camera (ASC) in the aftermath of the 2016 Kumamoto earthquake in Japan. It explains how the ASC was instrumental in providing crucial information to rescue teams, resulting in the successful rescue of multiple trapped occupants. The paper emphasizes the significance of utilizing advanced imaging technologies like the ASC in urban search and rescue operations, as they can improve the effectiveness and safety of rescue workers (Ambe et al., 2016)
Disaster response and recovery from the perspective of robotics
This paper presents a summary of the use of robotic operations in disaster scenarios. It examines the difficulties encountered by emergency responders and rescue teams in disaster-affected regions, including restricted access to the affected areas, dangerous conditions, and scarce resources (Park et al., 2017).
Drone-assisted disaster management: Finding victims via infrared camera and lidar sensor fusion
This article mentions that the use of drones has proven to be an efficient method to localize survivors in hard-to-reach areas (e.g., collapsed structures) (Lee et al., 2016). The paper presents a comprehensive framework for drone hardware that makes it possible to explore GPS-denied environments. Furthermore, the hokuyo lidar is used for global mapping, and the Intel RealSense for local mapping. The outcomes show that the combination of these sensors can assist USAR operations to find victims of natural disasters.
Boston Dynamics’ Spot Is Helping Chernobyl Move Towards Safe Decommissioning
This article presents the Boston Dynamics’ Spot robot. It discusses that the robot can be used to inspect and monitor the structural integrity of buildings (Ackerman, 2020). The robot has potential as it can perform dangerous tasks in hazardous environments which reduces the risk for human workers.
Rescue Robots for the Urban Earthquake Environment
The utilization of robots in USAR operations can decrease response time, enhance the effectiveness of rescue operations, and ensure the safety of USAR personnel (Li et al., 2022). However, when testing the robot in Italy, the robot operator experienced cognitive overload. The data provided by the robot was not intuitive, so much information still had to be managed by people. Consequently, rescue personnel with limited training were unable to effectively control the robot on site.
Earthquake sites
Rescue robotics: DDT project on robots and systems for urban search and rescue
Tremors caused by an earthquake destroy buildings, can generate tsunamis, and cause fires, landslides, etc. (Tadokoro, 2009). Due to this, inhabitants can be buried alive, burnt to death, or drowned. Immediate USAR is important as the survival rate decreases as time passes. Especially, the first 72 hours are important and called the golden 72 hours. “Many first responders state that they can rescue if the victim’s position is known. Often, search is beyond human ability” (Tadokoro, 2009, p. 11). This book names equipment used in USAR. Among others, these are an infrared camera to detect human survivors, microwave radar to detect heartbeats, and a fiber scope which is a bending camera.
USAR experiences
Urban Search and Rescue Team. Update zaterdagmiddag
Quote from the blog of the Dutch search team in Turkey: "We merken dat de honden vermoeid raken door de vele inzetten." (Urban Search and Rescue Team, 2023). This implements that dogs are getting tired after searching for survivors for several days.
Previous projects
PRE2015 2 Groep1 - Universal Swarm Robotics Software Project
Swarm robotics used for helping people in the debris after an earthquake. The robots communicate with each other to help each other in order to remove rubble.
This group presented drone robotics in order to seek for survivors. The drones were controlled via radio control on a 900 MHz frequency.
This project researched how a swarm of RoboBees (flying microbots) could be used in USAR operations. The main concern of this project was to identify the location of a gas leak and to communicate that information to the USAR team. They looked into infrared technology, wireless LAN and bluetooth for communication.
Problem Identification
The problem that will be addressed throughout this paper is the development of a cost-effective vine robot to be used in urban search and rescue (USAR) missions. These missions are known to be extremely critical, which require precise technology to navigate and locate survivors through complex terrain. However, current rescue robots have multiple limitations which make it difficult for them to be used successfully. As a result, this paper will explore the vine robot as an alternative solution.
To this day, vine robots have not been implemented into critical USAR missions, since they do not have any navigation or localization abilities. Therefore, research needs to be conducted to solve this problem and allow vine robots to be used in real-life rescue missions. The most suitable sensors will need to be identified for localization, as well as the preferred navigation strategy. On top of this, the level of autonomy will need to be analyzed for the vine robots application.
Overall, current rescue robots have have limitations such as mobility issues, limited sensing and limited autonomy. As a result, this paper will explore these problems and come up with a viable solution for the vine robot to be used in real-life USAR applications.
USE
Users
Within urban search and rescue operations, several users of the vine robot can be named.
First, the International Search and Rescue Advisory Group (INSARAG) determines the minimum international standards for urban search and rescue (USAR) teams (INSARAG – Preparedness Response, z.d.). This organization establishes a methodology for coordination in earthquake response. Therefore, this organization will have to weigh the pros and cons of using a vine robot in USAR. If INSARAG sees the added value of using robotics in search and rescue operations, it can promote the usage, and include it in the guidelines.
Second, governments will need to purchase all necessary equipment. For the Netherlands, Nationaal Instituut Publieke Veiligheid is the owner of all the equipment of the Dutch USAR team (Nederlands Instituut Publieke Veiligheid, 2023). This Institute will need to see the added value of the robot while taking into account the guidelines of INSARAG.
The third group of users consists of members of the USAR teams that will have to work with the vine robot on site. The vine robot will be used alongside other techniques that are already used right now. USAR teams are multidisciplinary and not all members of the team will come in contact with the robot (e.g., nurses or doctors). In order to properly use the vine robot, USAR members who execute the search and rescue operation will need training. For the Dutch USAR team, this training can be conducted by the Staff Officer Education, Training and Exercise (Het team - USAR.NL, z.d.). USAR members will need to be able to set up the vine robot, navigate it inside a collapsed building (if it is not fully autonomous), read data that the vine robot provides, and find survivors with the help of the vine robot. Furthermore, they will need to decide whether it is safe to follow the path of the vine robot to a survivor. Lastly, team members will need to retract the vine robot and reuse it if possible.
At last, the victims of the earthquake that the vine robot will be used for to localize. They will not have any control over the vine robot but will come into contact with it. It is therefore important that they will not be scared of the robot and will try to defend themselves from it.
Users' needs
In order to gather knowledge regarding the needs of these users, emails are sent out to several USAR teams, INSARAG, and Nationaal Instituut Publieke Veiligheid, containing the following questions:
- What type of sensors or technology do you use for localization?
- How do you localize survivors?
- If there are methods that allow robots to go closer within rubble, are there specific things to keep in mind for localization?
- What can your current equipment not do and what would you like them to improve on?
- What is the main issue you have on current equipment?
- What makes a rescue operation expensive?
- What is the protocol for when you are unable to rescue a survivor? (e.g. assigning probabilities to survivors as resources are limited)
Questions for Nationaal Instituut Publieke Veiligheid:
- How is it determined what material is bought for USAR operations?
- How do you feel about using robotics in USAR operations?
- What makes a rescue operation expensive?
Furthermore, similar questions were posted on Reddit.
Society
Society will benefit from vine robots as it will help USAR operations to localize victims and find a path within rubble to a victim. This will influence the time needed to search for survivors after earthquakes. This is important as the chances of surviving decrease with each passing day. Thus the usage of vine robots will influence the number of people that can be saved.
Enterprise
If the vine robot will be available for sale, the company behind it will have several interests. First, it will want to create a robot that can make a difference. The robot should help USAR operations with localizing survivors in the rubble. Second, the company will want to either make a profit or make enough money to break even. It will need money to invest back in the product to further improve the robot. For the company, it is important to take into account the guidelines of INSARAG as this institute will promote the usage of rescue robots in their global network.
Specifications
Before identifying the solutions for navigation and localization, a clear list of specifications for the vine robot is given.
Must:
- The vine robot must communicate from the tip of the vine, back to its starting location (the pump). This would allow the rescuers to communicate with the survivor by providing live feedback.
- For its application, the vine robot must be semi-autonomous. A human operator must move the vine robot by command, as well as the robot maneuvering itself when xxx???
- Sensors must be used for localizing the survivor. The exact sensors will be explored in the next chapter.
- Since the vine robot is to be used in critical rescue missions, it must be easy to transport and quick to setup. This would make it easier to increase the number of vine robots used in USAR missions, which increases the overall success rate in localizing survivors.
Should:
- Supply water and air
- Be relatively cheap and easy to manufacture
Could:
- Communicate with other vine robots (e.g. swarm robots)
- Autonomously find survivors (e.g. localization)
- Inflate into safety structure
- Retract back to its original path
- Create a 3D model of the environment
Won't:
- Actively get survivors out of situations (it only assists)
- Supply heating or other health supplies other than water or air
- Be infinitely long (it has a fixed length)
- Be able to lift x kg
- Be able to put out fires and melt in extreme heat
Now that the specifications have been determined, the constraints of the robot can be established. This mainly focuses on the design of the robot, such as the material and size.
Constraints:
- Size:
- Material (+ durability):
- Weight:
- Cost:
- Load Capacity (how much weight it can lift up)
- Mobility (e.g. smallest gap it can move through)
Pathfinding and localization
Current Difficulties
Earthquakes are one of the most devastating natural disasters, causing widespread destruction and loss of life. One of the biggest challenges that emergency responders and aid organizations face in the aftermath of an earthquake is localizing victims. This task can be extremely difficult due to a variety of factors, including the scale of the disaster, the nature of the terrain, and the complexity of the affected infrastructure.
Firstly, the scale of the disaster is often overwhelming, making it difficult for rescue teams to quickly locate and reach those in need of assistance. Earthquakes can cause extensive damage to buildings, roads, and other infrastructure, which can make it challenging for rescue teams to navigate the affected areas. Additionally, earthquakes can cause landslides, debris flows, and other hazardous conditions that can further impede rescue efforts.
Secondly, the nature of the terrain can also make it difficult to localize victims after earthquakes. Many earthquakes occur in mountainous or hilly areas, which can be challenging for rescue teams to access. These areas may have steep slopes, narrow paths, and other obstacles that can make it difficult to reach victims. Additionally, earthquakes can cause landslides and rockfalls, which can further complicate rescue efforts.
Thirdly, the complexity of the affected infrastructure can also pose challenges for rescue teams. Earthquakes can damage roads, bridges, and other infrastructure, which can make it difficult for rescue teams to access affected areas. In addition, damage to communication networks can make it difficult for rescue teams to coordinate their efforts and share information about the location of victims.
Lastly, the timing of earthquakes can also complicate rescue efforts. Earthquakes can occur at any time, day or night, and may cause power outages, making it difficult for rescue teams to operate in the dark. Additionally, aftershocks can further damage infrastructure and create additional hazards, making it difficult for rescue teams to work safely.
In conclusion, localizing victims after earthquakes is a challenging task that requires extensive planning, coordination, and resources. The scale of the disaster, the nature of the terrain, the complexity of the affected infrastructure, and the timing of the earthquake can all pose significant challenges for rescue teams.
Localization for vine robots
Localization of surviors is a critical task in search and rescue operations in destroyed buildings. Vine robots can play an important role in this task by using their flexibility and agility to navigate through complex and unpredictable environment and locate survivors. Localization involves determining the position of the robot and the position of any survivors in the enviroment and can be achieved through a variety of techniques and strategies.
An approach to localization is to use a combination of sensors and algorithms to detect and track the location of survivors. This can include sensors that detect sound, heat or movement as as algorithms that use this data to determine the location of survivors in the environment. What current technology has these capabilities?
Detecting of heat
Vine robots can be equipped with a range of sensors that enable them to detect heat in their surroundings. Infared cameras and thermal imaging systems are among the most commonly used sensors for detecting heat in robots.
Infared cameras work by detecting infared radiation emitted by objects and converting it into visual representations that can be interpreted by the robot's control systems.
Add diagram for infrared
Thermal imaging systems use a more advanced technology that can detect temperature changes with higher precision, which can enable vine robots to identify potential sources of heat and determine the location and movement of individuals in a given environment
Add diagram for thermal imaging
Other types of sensors that could be used in vine robots for detecting heat also include contact sensors or gas sensors.
Contact sensors can be used to detect heat sources that come into direct contact with the robot's sensors. For example, if a vine robot comes into contact with a hot object such as a stove or a piece of machinery, the heat from the object can be detected by the contact sensors.
Gas sensors can be used to detect the presence of combustible gases such as methane or propane which can be an indicator of a potential fire or explosion
Detecting of sound
Detecting sound is another critical capability for vine robots especially in search and rescue operations. Vine robots can be equipped with a range of sensors that enable them to detect and interpret sound waves in their surroundings. A common type sensor used to detect sound is a microphone. Microphones can be used to capture sound waves in the enviroment and convert them into electrical signal that can be interpreted by the robot's control systems or be communicated back to a human operator for analysis
Ultrasonic sensors
Vine robots can also be equipped with ultrasonic sensors which enable them to detect sound waves that are beyond the range of human hearing. Ultrasonic sensors work by emitting high-frequency sound waves that bounce off objects in the environment and return to the sensor, producing an electrical signal that be interpreted by the robot's control systems.
Vibration sensors
In addition to ultrasonic sensors, vine robots can be equipped with vibration sensors, which can detect sound waves that are not audible to the human ear. Vibration sensors work by detecting the tiny vibrations in solid objectes caused by sound waves passing through them. These vibrations are converted into electrical signals that can be interpreted by the robot's control systems.
Detecting movement
Detecting movement is another critical capability for vine robots, espcially in search and rescue operations. Vien robots can be equipped with a range of sensors that enable them to detect and interpret movement in their surroundings.
Cameras
One common type of sensor used in vine robots for detecting movement is a camera. Cameras can be used to capture visual data from the environment and interpret it using computer vision algorithms to detect movement.
Motion sensors
Another type of sensor used in vine robots for detecting movement is a motion sensors.
Motion sensors or motion detectors are electronic devices that are designed to detect movement in their surrounding environment. They work by measuring changes in the level of infrared radiation, sound waves or vibrations caused by movement.
There are 4 types of motion sensors being used in the industry:
- Passive infrared (PIR) sensors: These sensors detect changes in the level of infrared radiation (heat) in their field of view caused by moving objects.
- Ultrasonic sensors: These sensors emit high-frequency sound waves and measure the time it takes for the waves to bounce back after an object. If an object moves in front of the sensors, it will cause a change in the time it takes for the sound waves to bounce back, which triggers an response.
- Mircowave sensors: Similar to ultrasonic sensors. But these sensors emit electromagnetic waves instead.
- Vibration sensors: These sensors measure changes in acceleration caused by movement.
Add diagrams for detection types.
Pathfinding for vine robots
Pathfinding is the process of finding the shortest or most efficient path between two points in a given environment. For a vine robot, pathfinding is critical as it allows the robot to navigate throught the porous environment of destroyed buildings and reach its intended destination. Pathfinding algorithms are typically used to determine the best route for the robot to take based on factors such as obstacles, terrain and distance.
One approach is to use a combination of reactive and deliberative pathfinding strategies.Reactive pathfinding involves using sensors to detect obstacles in real-time and making rapid adjustments to the robot's path to avoid them. This can be especially useful in environments where the obstacles and terrain are constantly changing such as in a destroyed building. Deliberative pathfinding on the other hand involves planning paths ahead of time based on a map or model of the environment. While this approach can be useful in some cases, it may not be practical in a destroyed building where the environment is constantly changing.
Another approach is to use maching learning algorithms to train the robot to navigate through the enviroment based on real-world data.
Sensors and path planning
Colas et al. (2013) present a path planning that works for a 3D terrain. The system makes use of exteroceptive sensors, it uses a front-mounted rolling laser scanner that can take full three-dimensional (3D) scans of its surrounding. "This system is based on point cloud data and does not attempt to fully reconstruct the environment, but instead uses lazy tensor voting to assess traversability." Tensor voting means that it extracts geometrical primitives and saliency by voting of points So far, this system has been implemented for static environments, in dynamic environments it still has challenges. Which raises the question if our robot should be able to path plan in a dynamic environment and if this is useful for our robot.
When we look at the table provided we see use cases of how to detect people under debris, with the current technologies and how to continue research into them. Ferrara V. (2015), explains everything in the paper regarding the shortcomings and benefits of each solution. Giving us a good start to figure out what technologies we need to research. Provided below is some information about them, but each would need to be researched in depth. I think for the current project idea, one of the following would be enough to deepen in as the full research into one of these would already take too much time.
Constant false alarm rate (CFAR)
CFAR detection refers to a common form of adaptive algorithm used in radar systems to detect target returns against a background of noise, clutter and interference.
An important CFAR algorithm is the cell averaging (CA) CFAR, in which the mean background level is estimated by averaging the signal level in M neighboring range cells.
Self-injection-locked (SIL)
The SIL radar is operated at 433 MHz ISM band to achieve excellent penetration capability and coverage. Moreover, an additional phase shifter is utilized to eliminate the large frequency shift, which is caused by strong clutter signals and often causes the SIL mechanism to fail.
Ultra-wideband (UWB)
UWB has traditional applications in non-cooperative radar imaging. Most recent applications target sensor data collection, precise locating, and tracking. Ultra low-power radio-frequency identification (RFID) tag with precision localization is often the enabling technology for location-aware sensor applications. Impulse-Radio Ultra-Wideband (IR-UWB) is a promising technology to fulfill the usage requirements in indoor cluttered environment. This doesn't specifically mean that it will work well under interference or rubble for that matter, but it does show us some promising results.
| Methods (sources will be added later) | Localizing Victims under Rubble | Localizing Victims Fast | Precision |
|---|---|---|---|
| Visual Recognition with Rescue Dogs | No | Yes | High |
| Life Signs Detector Using a Drone in Disaster Zones | No | Yes | High |
| Audio-Processing-Based Human Detection in Disaster Sites with Unmanned Aerial Vehicle | Maybe | No | Low |
| Methods for Autonomous Wristband Placement with a Search-and-Rescue Aerial Manipulator | No | No | Precise |
| Victim Localization Using Bluetooth Low-Energy Sensors | Yes | Yes | Precise only at the surface |
| Detection and Location of Victims Using WiFi FTM and UWB | Yes | Yes | Precise |
| (more sources will come) |
Simulation
Goal
The goal of the simulation is to determine what algorithm is the best fit as our robot’s pathfinding algorithm and to prove the algorithm’s functionality. The algorithm can be split into two parts. First the vine robot has to find a sign of life by systematically searching through the debris. Then once the vine robot has picked up on such a sign of life, it must adjust its route to move towards the source of that sign of life in order to get the exact location of a survivor. The latter part of the algorithm is particularly of interest here, which is why we save this route. Aside, from relaying this information back, it’s also important for the stretch goal of having swarm robotics.
Specification
The simulation is made using NetLogo 3D. This software has been chosen for its simplicity in representing the environment using patches and turtles.
The vine robot is represented as a turtle. For algorithms involving swarm robotics, multiple turtles can be used. The robot can move (diagonally) forwards and to its (diagonal) sides.. It cannot move (diagonally) backwards. The robot cannot move onto patches it has visited before, because the vine robot would then hit itself. The robot can only see the patches towards which it can move. When involving swarm robotics we wouldn’t want them to cross each other either, thus by saving and relaying its path we can save on resources (vine robots don’t traverse the same path) and prevent them from crossing/crashing into each other.
The environment is randomly filled with large chunks of debris. These are represented by grey patches. The robot may not move onto these patches. To simulate the debris from collapsed buildings more closely, the debris patches are clustered together.
The environment also contains a survivor, which is represented by a stationary turtle. When within an adjustable range of the survivor, the robot can pick up on a sign of life. Once this happens, the robot can determine the direction of the source, but it cannot know the exact location until the target is reached. The algorithm used for getting to the target is of particular interest in this simulation. Once the robot has reached the survivor, the user is notified and the simulation is stopped.
Design choices
The simulation cannot represent the whole scenario. We had to make some design decisions where it may not be directly apparent as to why we chose them, thus some of our design choices will be explained further in detail here.
In the simulation, the survivor cannot be inside a patch that indicates a large chunk of debris. Intuitively this would mean that the survivor is stuck in small, light debris which the vine robot can move through. However, we know that the target is stuck underneath rubble, as they would not need rescuing otherwise. As such, it is assumed that the weight of all the rubble underneath which the survivor is trapped prevents them from moving into another patch. The robot, however, is able to move between patches of small, light debris due to its lesser size. Whether the weight is caused by a single big piece of debris or a collection of smaller pieces of debris can be influential in getting the survivor out of the debris, but it is not important for finding them. Thus it is also not important for the simulation.
Weekly breakdowns
| Name | Total | Breakdown week 1 |
|---|---|---|
| Clinton Emok | 3h | Meeting (1h), literature research (1h), user definition(1h) |
| Richard Farla | 4h | Brainstorm session (1h), meeting (1h), literature research (1h), milestones (1h) |
| Yash Israni | 3h | Meeting (1h), user requirements(1h), literature research (1h) |
| Tessa de Jong | 4h | Brainstorm session (1h), meeting (1h), problem statement (1h), literature research (1h) |
| Kaj Scholer | 4h | Brainstorm session (1h), meeting (1h), milestones (1h), literature research (1h) |
| Pepijn Tennebroek | 4h | Brainstorm session (1h), meeting (1h), problem statement (1h), literature research (1h) |
| Name | Total | Breakdown week 2 |
|---|---|---|
| Clinton Emok | 6h | Meeting 1 (2h), Meeting 2 (2h), Meeting 3 (1h), Localization (1h) |
| Richard Farla | 6h | Meeting 1 (2h), Meeting 2 (2h), Research equipment + how to build (2h) |
| Yash Israni | 7h | Meeting 1 (2h), Meeting 2 (2h), Meeting 3 (1h), Sensors (2h) |
| Tessa de Jong | 7h | Meeting 1 (2h), Meeting 2 (2h), Meeting 3 (1h), Literature research (2h) |
| Kaj Scholer | 10.5 | Meeting 1 (2h), Meeting 2 (2h), Meeting 3 (1h), CAD Modelling (2h), Storyboard (2.5h), Primary Reddit Research (1h) |
| Pepijn Tennebroek | 7h | Meeting 1 (2h), Meeting 2 (2h), Meeting 3 (1h), Fix literature (1h), Research Pathplanning (2h) |
| Name | Total | Breakdown week 3 |
|---|---|---|
| Clinton Emok | 7h | Meeting 1 (2h), Meeting 2 (1h), Meeting 3 (1h), Writing paper (1h), Research(1h), State-of-the-art(1h) |
| Richard Farla | 7h | Meeting 1 (2h), Simulation (3h), Meeting 2 (1h), Meeting 3 (1h) |
| Yash Israni | 6h | Simulation (1h), Meeting 2 (1h), Meeting 3 (1h), Researching papers (3h) |
| Tessa de Jong | 9h | Meeting 1 (2h), Problem statement and Users (3h), Meeting 2 (1h), Meeting 3 (1h), Research (2h) |
| Kaj Scholer | 8h | Meeting 1 (2h), Meeting 2 (1h), Meeting 3 (1h), Fixing Wiki Structure/Content (1h), State-of-the-art (1h), Localization + Sensors (2h) |
| Pepijn Tennebroek | 8h | Problem statement and Users (3h), Meeting 2 (1h), Fix literature (3h) Meeting 3 (1h) |
| Name | Total | Breakdown week 4 |
|---|---|---|
| Clinton Emok | Meeting 1 (2h), Localization/Sensors (3h), Meeting 2 (30min), Meeting 3 () | |
| Richard Farla | Meeting 1 (2h), Simulation (5h), Meeting 2 (30min), Meeting 3 () | |
| Yash Israni | Meeting 1 (2h), Simulation (1.5h), Meeting 2 (30min), Meeting 3 () | |
| Tessa de Jong | Meeting 1 (2h), State-of-the-art (3h), USE (3h), Meeting 2 (30min), Meeting 3 () | |
| Kaj Scholer | Meeting 1 (2h), Meeting 2 (30min), Problem statement (1h), Specifications (1h), Meeting 3 () | |
| Pepijn Tennebroek | Meeting 1 (2h), State-of-the-art (2h), USE (2h), Meeting 2 (30min), Meeting 3 () |
References
AJLabs. (2023). Infographic: How big were the earthquakes in Turkey, Syria? Earthquakes News | Al Jazeera. https://www.aljazeera.com/news/2023/2/8/infographic-how-big-were-the-earthquakes-in-turkey-syria
Blumenschein, L. H., Coad M. M., Haggerty D. A., Okamura A. M., & Hawkes E. W. (2020). Design, Modeling, Control, and Application of Everting Vine Robots. https://doi.org/10.3389/frobt.2020.548266
Coad, M. M., Blumenschein, L. H., Cutler, S., Zepeda, J. A. R., Naclerio, N. D., El-Hussieny, H., Mehmood, U., Ryu, J., Hawkes, E. W., & Okamura, A. M. (2020). Vine Robots: Design, Teleoperation, and Deployment for Navigation and Exploration. https://arxiv.org/pdf/1903.00069.pdf
Urban Search and Rescue Team. (2023). Update zaterdagmiddag. USAR. https://www.usar.nl/
Zepeda, J. A. R. (2022). The 1-minute vine robot. Vine Robots. https://www.vinerobots.org/build-one/the-simple-vine-robot/
Zotomayor, C., (2021). This Vine Robot Is an Unstoppable Force With Tons of Applications. https://www.solidsmack.com/design/vine-robot/
F. Colas, S. Mahesh, F. Pomerleau, M. Liu and R. Siegwart, "3D path planning and execution for search and rescue ground robots," 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, Tokyo, Japan, 2013, pp. 722-727, doi: 10.1109/IROS.2013.6696431. This paper presents a pathplanning system for a static 3D environment, with the use oflazy tensor voting.
Tai, Y.; Yu, T.-T. Using Smartphones to Locate Trapped Victims in Disasters. Sensors 2022, 22, 7502. https://doi.org/10.3390/ s22197502
Da Hu, Shuai Li, Junjie Chen, Vineet R. Kamat,Detecting, locating, and characterizing voids in disaster rubble for search and rescue, Advanced Engineering Informatics, Volume 42,2019,100974,ISSN 1474-0346, https://doi.org/10.1016/j.aei.2019.100974
Park, S., Oh, Y., & Hong, D. (2017). Disaster response and recovery from the perspective of robotics. International Journal of Precision Engineering and Manufacturing, 18(10), 1475–1482. https://doi.org/10.1007/s12541-017-0175-4
Raibert, M. H. (2000). Legged Robots That Balance. MIT Press
Tenreiro Machado, J. A., & Silva, M. F. (2006). An Overview of Legged Robots
Murphy, R. R. (2017). Disaster Robotics. Amsterdam University Press
Sanfilippo F, Azpiazu J, Marafioti G, Transeth AA, Stavdahl Ø, Liljebäck P. Perception-Driven Obstacle-Aided Locomotion for Snake Robots: The State of the Art, Challenges and Possibilities †. Applied Sciences. 2017; 7(4):336. https://doi.org/10.3390/app7040336
Kruijff, G. M., Kruijff-Korbayová, I., Keshavdas, S., Larochelle, B., Janíček, M., Colas, F., Liu, M., Pomerleau, F., Siegwart, R., N., Looije, R., Smets, N. J. J. M., Mioch, T., Van Diggelen, J., Pirri, F., Gianni, M., Ferri, F., Menna, M., Worst, R., . . . Hlaváč, V. (2014). Designing, developing, and deploying systems to support human–robot teams in disaster response. Advanced Robotics, 28(23), 1547–1570. https://doi.org/10.1080/01691864.2014.985335
Boston Dynamics, Inc. (2019). Robotically negotiating stairs (Patent Nr. 11,548,151). Justia. https://patents.justia.com/patent/11548151
Nosirov, K. K., Shakhobiddinov, A. S., Arabboev, M., Begmatov, S., and Togaev, O.T. (2020) "Specially Designed Multi-Functional Search And Rescue Robot," Bulletin of TUIT: Managementand Communication Technologies: Vol. 2 , Article 1. https://doi.org/10.51348/tuitmct211
Delmerico, J., Mintchev, S., Giusti, A., Gromov, B., Melo, K., Horvat, T., Cadena, C., Hutter, M., Ijspeert, A., Floreano, D., Gambardella, L. M., Siegwart, R., & Scaramuzza, D. (2019). The current state and future outlook of rescue robotics. Journal of Field Robotics, 36(7), 1171–1191. https://doi.org/10.1002/rob.21887
Al-Naji, A., Perera, A. G., Mohammed, S. L., & Chahl, J. (2019). Life Signs Detector Using a Drone in Disaster Zones. Remote Sensing, 11(20), 2441. https://doi.org/10.3390/rs11202441
Uddin, Z., & Islam, M. (2016). Search and rescue system for alive human detection by semi-autonomous mobile rescue robot. 2016 International Conference on Innovations in Science, Engineering and Technology (ICISET). https://doi.org/10.1109/iciset.2016.7856489
Liu, Y., Nejat, G. Robotic Urban Search and Rescue: A Survey from the Control Perspective. J Intell Robot Syst 72, 147–165 (2013). https://doi.org/10.1007/s10846-013-9822-x
Anthes, Gary. Robots Gear Up for Disaster Response. Communications of the ACM (2010): 15, 16. Web. 10 Oct. 2012
Osumi, H. (2014). Application of robot technologies to the disaster sites. Report of JSME Research Committee on the Great East Japan Earthquake Disaster, 58-74
Matsuno, F., Sato, N., Kon, K., Igarashi, H., Kimura, T., Murphy, R. (2014). Utilization of Robot Systems in Disaster Sites of the Great Eastern Japan Earthquake. In: Yoshida, K., Tadokoro, S. (eds) Field and Service Robotics. Springer Tracts in Advanced Robotics, vol 92. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-40686-7_
Kawatsuma, S., Fukushima, M., & Okada, T. (2013). Emergency response by robots to Fukushima-Daiichi accident: summary and lessons learned. Journal of Field Robotics, 30(1), 44-63. doi: 10.1002/rob.21416
Hatazaki, K., Konyo, M., Isaki, K., Tadokoro, S., and Takemura, F. (2007) Active scope camera for urban search and rescue IEEE/RSJ International Conference on Intelligent Robots and Systems, San Diego, CA, USA, 2007, pp. 2596-2602, doi: 10.1109/IROS.2007.4399386
Ambe, Y., Yamamoto, T., Kojima, S., Takane, E., Tadakuma, K., Konyo, M., & Tadokoro, S. (2016). Use of active scope camera in the Kumamoto Earthquake to investigate collapsed houses. International Symposium on Safety, Security, and Rescue Robotics. https://doi.org/10.1109/ssrr.2016.7784272
Zhao, L., Sun, G., Li, W., & Zhang, H. (2016). The design of telescopic universal joint for earthquake rescue robot. 2016 Asia-Pacific Conference on Intelligent Robot Systems (ACIRS). https://doi.org/10.1109/acirs.2016.7556189
Kamezaki, M., Ishii, H., Ishida, T., Seki, M., Ichiryu, K., Kobayashi, Y., Hashimoto, K., Sugano, S., Takanishi, A., Fujie, M. G., Hashimoto, S., & Yamakawa, H. (2016). Design of four-arm four-crawler disaster response robot OCTOPUS. International Conference on Robotics and Automation. https://doi.org/10.1109/icra.2016.7487447
De Cubber, G., Doroftei, D., Serrano, D., Chintamani, K., Sabino, R., & Ourevitch, S. (2013, October). The EU-ICARUS project: developing assistive robotic tools for search and rescue operations. In 2013 IEEE international symposium on safety, security, and rescue robotics (SSRR) (pp. 1-4). IEEE.
Ackerman, E. (2023i). Boston Dynamics’ Spot Is Helping Chernobyl Move Towards Safe Decommissioning. IEEE Spectrum. https://spectrum.ieee.org/boston-dynamics-spot-chernobyl
Lee, S., Har, D., & Kum, D. (2016). Drone-Assisted Disaster Management: Finding Victims via Infrared Camera and Lidar Sensor Fusion. 2016 3rd Asia-Pacific World Congress on Computer Science and Engineering (APWC on CSE). https://doi.org/10.1109/apwc-on-cse.2016.025
Lindqvist, B., Karlsson, S., Koval, A., Tevetzidis, I., Haluška, J., Kanellakis, C., Agha-mohammadi, A. A., & Nikolakopoulos, G. (2022). Multimodality robotic systems: Integrated combined legged-aerial mobility for subterranean search-and-rescue. Robotics and Autonomous Systems, 154, 104134. https://doi.org/10.1016/j.robot.2022.104134
Li, F., Hou, S., Bu, C., & Qu, B. (2022). Rescue Robots for the Urban Earthquake Environment. Disaster Medicine and Public Health Preparedness, 17. https://doi.org/10.1017/dmp.2022.98
Tadokoro, S. (Ed.). (2009). Rescue robotics: DDT project on robots and systems for urban search and rescue. Springer Science & Business Media.
Hampson, M. (2022). Detecting Earthquake Victims Through Walls. IEEE Spectrum. https://spectrum.ieee.org/dopppler-radar-detects-breath
Huamanchahua, D., Aubert, K., Rivas, M., Guerrero, E. L., Kodaka, L., & Guevara, D. C. (2022). Land-Mobile Robots for Rescue and Search: A Technological and Systematic Review. 2022 IEEE International IOT, Electronics and Mechatronics Conference (IEMTRONICS). https://doi.org/10.1109/iemtronics55184.2022.9795829
Bangalkar, Y. V., & Kharad, S. M. (2015). Review Paper on Search and Rescue Robot for Victims of Earthquake and Natural Calamities. International Journal on Recent and Innovation Trends in Computing and Communication, 3(4), 2037-2040.
Localization/Sensors
Agarwal, T. (2019, July 25). Vibration Sensor: Working, Types and Applications. ElProCus - Electronic Projects for Engineering Students. https://www.elprocus.com/vibration-sensor-working-and-applications/#:~:text=The%20vibration%20sensor%20is%20also,changing%20to%20an%20electrical%20charge
Agarwal, T. (2022, May 23). Different Types of Motion Sensors And How They Work. ElProCus - Electronic Projects for Engineering Students. https://www.elprocus.com/working-of-different-types-of-motion-sensors/
Appendix
User Study 1
Query:
I am currently doing a project about technology being used in search and rescue situations. As a group, we have come up with some questions. These questions are here to give us an idea on areas which can be improved, such as the problems with current equipment. If you have contact with other SAR organizations, we would be grateful if you can help us get into contact with them, so that we can further our analysis. Thank you all in advance!
- What type of sensors/technology are currently used to localize a survivor?
- What are the main issues with the current equipment or what do they lack in?
- What makes a rescue operation expensive?
- What are some protocols for when a survivor cannot be rescued?
Results:
