PRE2024 3 Group7

Group

Name	Student ID	Email	Major	Ideas
Isaak Christou	1847260	i.christou@student.tue.nl	Electrical Engineering	Agricultural Weeding Robot Smart Trash Bin with AI Sorting (or no AI) Firefighting Drone Smart Walking Cane for the Visually Impaired Robotic Road Crack Detector
Luca van der Wijngaart	1565923	l.y.v.d.wijngaart@student.tue.nl	Computer Science
Daniel Morales Navarrete	1811363	d.morales.navarrete@student.tue.nl	Applied Mathematics	Robotic Beach Cleaner
Jeremiah Kamidi	1778013	j.j.kamidi@student.tue.nl	Psychology and Technology	self driving cars
Joshua Duddles	1719823	j.m.duddles@student.tue.nl	Psychology and Technology

The smart trash bin that auto sorts is a fun idea kind of like the machines used in industries where fast automatic sorting is required (for example sorting potatoes by colour, size, shape etc). This can replace all the bins on campus for example since lots of people don't use them correctly.

Robotic crack detector also looks fun. A prototype could be a cheap RC car with some sensors on it to accurately detect cracks and map out the roads. The city management could use it to better map out faults in roads and pavements that could hinder people.

Problem Statement

Road infrastructure is an essential component for economic growth in any country especially developing ones. Nevertheless the formation of cracks and potholes is common due to a multitude of factors such as weathering, excessive use, inferior design or materials etc. This makes road maintenance a significant challenge for any governing system as traditional methods for detecting such hazards often rely on visual inspection by personnel which is costly and time consuming or depends on the information provided by the general public which more often than not gets overlooked by authorities. With the increase of urbanization and the aging of road networks a more general, automated, cost effective and time efficient solution is required. A crack and pothole detection robot could help to quickly and accurately detect defects which can be quickly fixed reducing the overall cost and burden of the problem if left unchecked.

Our goal is to research on the current state of such technology and find better and more practical approaches. The research will be split into three parts: the carrier which is the form the robot takes for transportation (for example is a drone better than a grounded vehicle), the detection system (what hardware and software give a combination of best performance and cost and how can these also assess the state of the damage) and finally communication (how does the robot relay what it has found to a central hub that marks the location and state of the damage). Communication is especially important as allowing multiple agents to relay information is key in reducing the time needed to map large urban centres.

Objectives

• Examine and determine the ethical implications of autonomous road maintenance and data reaping.
• Determine how cracks in road surfaces can be detected using sensors, imaging, or other methods.
• Determine procedures for assessing the severity of cracks detected and classifying them for maintenance planning.
• Develop a model of the robotic crack inspection system.
• Develop a model for automated crack detection and classification using AI or other methodologies.
• Illustrate the system's usefulness through testing and comparison with traditional manual inspection procedures.

Approach, milestones and deliverables

Our approach to reaching these objectives in regards to the problem statement and the user needs is as follows:

We want to research and design a conceptual framework of a robot that is able to detect road cracks and map these out on a geographical map, for the benefit of road maintenance and infrastructure longevity. We will (partly) perform the first cycle of a multi-phase development cycle consisting of the following phases:

• Research & requirements gathering
  • This includes both research in the technology needed to build this robot but also the societal and economic effects of the robot and road maintenance in general.
• Ethical and legal considerations
• Sensor and other hardware selection
• Explain the road-crack detection and classification model
• Build a model/PoC (Proof of Concept)

Along these phases of our first development cycle we will set some milestones for ourselves as to keep our attention on the objectives set. This will be in the form of documentation of our work in a structured manner, making sure that the work put into each phase is represented. This documentation will in turn be part of our deliverables, as will be the model/PoC of our road-crack detecting robot.

Planning

who is doing what and general timeline

Everyone please fill in the task division that we just discussed. I think this section is also meant for the planning of the rest of project and I guess it's highly dependent on our Approach, milestones and deliverables so I'll also think about this.

Deadline	Student	Responsible topic/chapter	Done
15-02	Luca	Approach, milestone and deliverables
15-02	Dani	Objectives and 5 sources for State of the Art
15-02	Isaak	Problem Statement and 5 relevant papers

State of the art

3D vision technologies for a self-developed structural external crack damage recognition robot

This papers discusses the viability of multiple 3D vision techniques for detecting external cracks in infrastructure. This includes image based methods that only recently gained some adaptability, point cloud based methods that require substantial computational resources and 3D visual sensing and measuring methods such as 3D reconstruction. According to the article all methods presented lack one of three things: weight (the technology is usually to heavy), precision (to 0.1mm accuracy required for diagnosis) and robustness and accuracy. The authors then go to present a new type of automatic structural 3D crack detection system based on the fusion of high-precision LiDAR and camera which is more lightweight combines the depth sensing of LiDAR with the detailed imagery of the camera and has the required real time precision for safety diagnostics.

ROAD: Robotics-Assisted Onsite Data Collection and Deep Learning Enabled Robotic Vision System for Identification of Cracks on Diverse Surfaces

This paper discusses the architecture of ROAD (Robotics-Assisted Onsite Data Collection System) as a means of automatically detecting cracks and defects in road infrastructures. The paper looks into traditional methods of crack detection and their limitations and encourages the use of deep learning in crack detection. The paper also discusses the effectiveness of multiple deep learning algorithms in detecting cracks on roads and concludes that Xception has the best performance with an accuracy over 90% and mean square error of 0.03. More generally the paper claims that deep learning algorithm trained in good datasets outperform the traditional methods. The reason why the authors push for ROAD is due to the lack of automation when it comes to traditionally detecting cracks in roads and therefore introduces ROAD (Robotics-Assisted Onsite Data Collection System), which integrates robotic vision, deep learning, and Building Information Modeling (BIM) for real-time crack detection and structural assessment.

Novel pavement crack detection sensor using coordinated mobile robots

The paper proposes the design of an integrated unmanned ground vehicle (UGV) and drone system for real-time road crack detection and pavement monitoring. A drone conducts an initial survey using image analysis to locate potential cracks, while the UGV follows a computed path for detailed inspection using thermal and depth cameras. The collected data is processed using MATLAB and CrackIT, enhanced by a tailored image processing pipeline for improved accuracy and recall. A crowd-sourced crack database was developed to train and validate the system. Webots software was used for simulation, demonstrating the system’s effectiveness in structural health monitoring. The proposed system offers high mobility, precision, and efficiency, making it suitable for smart city applications.

Pixel-Wise Crack Detection Using Deep Local Pattern Predictor for Robot Application

This study introduces a novel crack detection method using a Convolutional Neural Network (CNN)-based Local Pattern Predictor (LPP). Unlike traditional methods that classify patches, this approach evaluates each pixel’s probability of belonging to a crack based on its local context. The proposed seven-layer CNN extracts spatial patterns, making the method robust to noise, lighting variations, and image degradation. Experiments using real-world bridge crack images demonstrate superior accuracy over existing methods (STRUM and block-wise CNN). The study also explores optimized sampling techniques and Fisher criterion-based training to enhance performance when datasets are limited. The method shows potential for real-time crack detection in robotic vision applications.

Development of AI- and Robotics-Assisted Automated Pavement-Crack-Evaluation System

The paper presents AMSEL, a semi-automated robotic platform designed to inspect pavement cracks in real-time using a deep learning model called RCDNet. The system uses both manual and automated navigation to collect data indoors and outdoors, with RCDNet detecting cracks based on image analysis. Despite some limitations, such as difficulty detecting cracks smaller than 1 mm and issues with lighting and shadow interference, the system provides an efficient alternative to manual inspections. Future improvements include integrating non-destructive testing (NDE) sensors, expanding the use of visual sensors for faster coverage, and developing deep learning models that can fuse data from multiple sources for more comprehensive defect detection.

Robotic surface exploration with vision and tactile sensing for cracks detection and characterization

The paper Robotic Surface Exploration with Vision and Tactile Sensing for Cracks Detection and Characterization suggests a hybrid approach to crack detection by complementing vision-based detection with tactile sensing. The system first employs a camera and object detection algorithm to identify potential cracks and generate a graph model of their structure. A minimum spanning tree algorithm then plans an effective exploration path for a robotic manipulator that reduces redundant movements.

To improve the accuracy of detection, a fiber-optic tactile sensor mounted on the manipulator verifies the presence of cracks, removing false positives from lighting or surface textures. Once verified, the system performs an in-depth characterization of the cracks, pulling out significant attributes such as length, width, orientation, and branching patterns. The two-sensing modality yields more precise measurements than traditional vision-only methods.

Experimental validation demonstrates that this integrated approach significantly enhances detection accuracy while reducing operating costs. By optimizing motion planning and reducing reliance on full-surface scanning, the system offers a more efficient and less expensive method of automated infrastructure inspection and maintenance.

Complete and Near-Optimal Robotic Crack Coverage and Filling in Civil Infrastructure

The paper Complete and Near-Optimal Robotic Crack Coverage and Filling in Civil Infrastructure proposes a new approach for autonomous crack inspection and repair with a simultaneous sensor-based inspection and footprint coverage (SIFC) planning scheme. The method blends real-time crack mapping and robot motion planning for effective and complete inspection. Integration of sensing and actuation through sensing and actuation integration makes the system efficient by avoiding redundant motion and providing optimal crack coverage.

The robot takes a two-step strategy, first, onboard sensors are used to detect and map cracks in real-time and calculate an optimal path of coverage using a greedy exploration algorithm. Second, a robotic manipulator follows the path and dispenses crack-filling substances where needed. The algorithm adjusts its path in real-time based on new cracks, allowing the system to react to irregular and complex surfaces without pre-computed structural maps.

Experimental results reveal that this system significantly improves the detection and effectiveness of crack repairs at a lower cost of operation. Through ensuring total crack coverage with minimal travel distance, the system outshines traditional procedures, making it a promising alternative for extensive rehabilitation of infrastructure.

Crack-pot: Autonomous Road Crack and Pothole Detection

The paper Crack-Pot: Autonomous Road Crack and Pothole Detection proposes an autonomous real-time road crack and pothole detection system using deep learning. This system employs a neural network architecture to handle road surface textures and spatial features, enabling the discrimination between damaged and undamaged areas. The approach improves the accuracy by reducing the misclassification due to environmental factors like lighting variations and surface unevenness.

The detection is carried out by capturing road images through a camera-based system mounted on an automobile or robotic platform. The images are input into a convolutional neural network (CNN) which identifies cracks and potholes based on their unique structural features. Compared with traditional thresholding-based methods, the learning-based approach is made versatile under different conditions with better robustness against occlusions, shadows, and background noise.

Experimental results show that the system achieves high accuracy of detection while operating in real-time, making it feasible for monitoring large-scale infrastructure. By automating road inspection, this method enhances efficiency and reduces the need for manual inspections, resulting in more proactive and cost-effective road maintenance procedures.

Visual Detection of Road Cracks for Autonomous Vehicles Based on DeepLearning

The research article Visual Detection of Road Cracks for Autonomous Vehicles Based on Deep Learning and Random Forest Classifier presents a high-tech image-based approach towards detecting road cracks based on the combination of deep learning and machine learning methods. The study integrates convolutional neural networks (CNNs) with a Random Forest classifier to improve accuracy in identifying faults in road surfaces. The method is intended to assist autonomous cars in driving over faulty roads while contributing to the maintenance of the infrastructure as well.

The system utilizes three state-of-the-art CNN models: MobileNet, InceptionV3, and Xception, trained on a 30,000 road image dataset. The learning rate of the network was tuned in experimentation to 0.001, yielding a maximum validation accuracy of 99.97%. The model was also tested on 6,000 additional images, where it recorded a high detection accuracy of 99.95%, demonstrating robustness under real-world conditions.

The results demonstrate the hybrid deep learning and machine learning technique significantly enhances crack detection accuracy compared to traditional methods. With its integration into autonomous vehicle technology or roadway maintenance initiatives, the technique offers a highly scalable, effective solution for real-time infrastructure monitoring and defect detection.

Article Deep Learning Based Pavement Inspection Using Self-Reconfigurable Robot

The paper Deep Learning-Based Pavement Inspection Using Self-Reconfigurable Robot introduces a robot system utilizing deep learning to conduct real-time pavement inspection and defect detection. The robotic system is centered on Panthera, a self-reconfigurable robot utilizing semantic segmentation and deep convolutional neural networks (DCNNs) for the detection of road defects and environmental obstructions such as litter.

The inspection process has two primary components: SegNet, a deep learning model that delineates pavement areas from other objects, and a DCNN-based defect detection module that detects different types of road defects. To enhance the system's usability in practical applications, it is integrated with a Mobile Mapping System (MMS) that geotags cracks and defects detected, allowing for precise location tracking. The Panthera robot has NVIDIA GPUs, which enable real-time processing and decision-making functions.

Experimental testing confirms that the system is highly accurate in detecting pavement damage and functions well under diverse urban environments. The technique not only optimizes the effectiveness of autonomous road maintenance and cleaning but also provides a scalable means for intelligent infrastructure management, reducing the need for manual inspections.

[11] The Current Opportunities and Challenges in Drone Technology

This recently published paper discusses the the advancements that Drone Sensor Technology and Drone Communcation Systems have made, after which it defines some opportunities and challenges that the field of drone technology faces and it draws some conclusions on where it thinks this technology is headed and the general importance this field will have in certain industries.

It discusses the applications of Drone technology in the sectors Agriculture, Healthcare, and Military & Security. According to the paper, drones have already started being a critical too in the Agriculture sector as they perform crop monitoring and analysis to detect diseases early, leading to improved yields, but also Livestock monitoring by tracking movements and using thermal cameras. Healthcare has started using drones for medical supply deliveries and emergency response: drones can easily get crucial supplies to hard-to-reach areas. Drones are also being used by the military for surveillance and reconnaissance and higher precision of (air-)strikes. This leads to less colateral damage and enhances battlefield efficiency.

The paper states some opportunities pertaining to these previously mentioned sectors, but more interestingly it states some challenges that it believes drone technology faces, that can be important to many sectors besides these 3. It mentions that current regulations and legal frameworks limit the use of drones immensely, and drones are prone to cybersecurity threats, being at the risk of hacking and unauthorized control. It also names some technical limitations such as limited battery life, payload capacity and drone costs being high.

[12] THz band drone communications with practical antennas: Performance under realistic mobility and misalignment scenarios

This recently published paper explores the role of Terahertz (THz) band communications in 6G non-terrestrial networks (NTN), focusing on drone-based connectivity, spectrum allocation, and power optimization. Drones are expected to act as airborne base stations, enabling high-speed, ultra-reliable connectivity for applications like surveillance, sensing, and localization.

The study evaluates the true performance of THz drone links under real mobility conditions and beam misalignment, finding that while data rates of 10s to 100s of Gbps are achievable, severe performance degradation can occur due to misalignment and antenna orientation changes. It analyzes three channel selection schemes (MaxActive, Common Flat Band, and Standard) along with two power allocation strategies (Water-Filling and Equal Power), identifying a commonly available THz band for stable transmission.

The paper highlights major challenges for THz drone communications, including frequency selectivity, beam misalignment, and mobility-induced disruptions. It emphasizes the need for active beam control solutions to maintain reliable performance. While THz technology offers vast bandwidth potential, overcoming alignment and stability issues is critical for practical deployment in 6G drone networks.

Work Records

Week 1

Name	Hours	Work
Isaak Christou	8	Made the wiki page, problem statement, 5 relevant papers summaries in state of the art
Luca van der Wijngaart	3	Group meeting, first start to the Approach section. first 2 out of 5 papers for state of the art section.
Daniel Morales	6 hours	Wrote objectives, found 5 relevant papers for state of the art and wrote a summary

Week 2

Week 2

Week 3

Week 3

Week 4

Week 4

Week 5

Week 5

Week 6

Week 6

Week 7

Week 7

Week 8

Week 8

References

[1][2][3][4][5]

[11] https://ijees.org/index.php/ijees/article/view/47

[12] https://www.sciencedirect.com/science/article/abs/pii/S1570870524002555