PRE2024 3 Group7
Group
| Name | Student ID | Major | Ideas | |
|---|---|---|---|---|
| Isaak Christou | 1847260 | i.christou@student.tue.nl | Electrical Engineering | Agricultural Weeding Robot
Smart Trash Bin with AI Sorting (or no AI) Firefighting Drone Smart Walking Cane for the Visually Impaired Robotic Road Crack Detector |
| Luca van der Wijngaart | 1565923 | l.y.v.d.wijngaart@student.tue.nl | Computer Science | |
| Daniel Morales Navarrete | 1811363 | d.morales.navarrete@student.tue.nl | Applied Mathematics | Robotic Beach Cleaner |
| Jeremiah Kamidi | 1778013 | j.j.kamidi@student.tue.nl | Psychology and Technology | self driving cars |
| Joshua Duddles | 1719823 | j.m.duddles@student.tue.nl | Psychology and Technology |
The smart trash bin that auto sorts is a fun idea kind of like the machines used in industries where fast automatic sorting is required (for example sorting potatoes by colour, size, shape etc). This can replace all the bins on campus for example since lots of people don't use them correctly.
Robotic crack detector also looks fun. A prototype could be a cheap RC car with some sensors on it to accurately detect cracks and map out the roads. The city management could use it to better map out faults in roads and pavements that could hinder people.
Problem Statement
Road infrastructure is an essential component for economic growth in any country especially developing ones. Nevertheless the formation of cracks and potholes is common due to a multitude of factors such as weathering, excessive use, inferior design or materials etc. This makes road maintenance a significant challenge for any governing system as traditional methods for detecting such hazards often rely on visual inspection by personnel which is costly and time consuming or depends on the information provided by the general public which more often than not gets overlooked by authorities. With the increase of urbanization and the aging of road networks a more general, automated, cost effective and time efficient solution is required. A crack and pothole detection robot could help to quickly and accurately detect defects which can be quickly fixed reducing the overall cost and burden of the problem if left unchecked.
Our goal is to research on the current state of such technology and find better and more practical approaches. The research will be split into three parts: the carrier which is the form the robot takes for transportation (for example is a drone better than a grounded vehicle), the detection system (what hardware and software give a combination of best performance and cost and how can these also assess the state of the damage) and finally communication (how does the robot relay what it has found to a central hub that marks the location and state of the damage). Communication is especially important as allowing multiple agents to relay information is key in reducing the time needed to map large urban centres.
Objectives
- Examine and determine the ethical implications of autonomous road maintenance and data reaping.
- Determine how cracks in road surfaces can be detected using sensors, imaging, or other methods.
- Determine procedures for assessing the severity of cracks detected and classifying them for maintenance planning.
- Develop a model of the robotic crack inspection system.
- Develop a model for automated crack detection and classification using AI or other methodologies.
- Illustrate the system's usefulness through testing and comparison with traditional manual inspection procedures.
Users (and what they need)
Approach, milestones and deliverables
Our approach to reaching these objectives in regards to the problem statement and the user needs is as follows:
We want to research and design a conceptual framework of a robot that is able to detect road cracks and map these out on a geographical map, for the benefit of road maintenance and infrastructure longevity. We will (partly) perform the first cycle of a multi-phase development cycle consisting of the following phases:
- Research & requirements gathering
- This includes both research in the technology needed to build this robot but also the societal and economic effects of the robot and road maintenance in general.
- Ethical and legal considerations
- Sensor and other hardware selection
- Explain the road-crack detection and classification model
- Build a model/PoC (Proof of Concept)
Along these phases of our first development cycle we will set some milestones for ourselves as to keep our attention on the objectives set. This will be in the form of documentation of our work in a structured manner, making sure that the work put into each phase is represented. This documentation will in turn be part of our deliverables, as will be the model/PoC of our road-crack detecting robot.
Planning
who is doing what and general timeline
Everyone please fill in the task division that we just discussed. I think this section is also meant for the planning of the rest of project and I guess it's highly dependent on our Approach, milestones and deliverables so I'll also think about this.
| Deadline | Student | Responsible topic/chapter | Done |
|---|---|---|---|
| 15-02 | Luca | Approach, milestone and deliverables | |
| 15-02 | Dani | Objectives and 5 sources for State of the Art | |
| 15-02 | Isaak | Problem Statement and 5 relevant papers |
State of the art
3D vision technologies for a self-developed structural external crack damage recognition robot
This papers discusses the viability of multiple 3D vision techniques for detecting external cracks in infrastructure. This includes image based methods that only recently gained some adaptability, point cloud based methods that require substantial computational resources and 3D visual sensing and measuring methods such as 3D reconstruction. According to the article all methods presented lack one of three things: weight (the technology is usually to heavy), precision (to 0.1mm accuracy required for diagnosis) and robustness and accuracy. The authors then go to present a new type of automatic structural 3D crack detection system based on the fusion of high-precision LiDAR and camera which is more lightweight combines the depth sensing of LiDAR with the detailed imagery of the camera and has the required real time precision for safety diagnostics.
ROAD: Robotics-Assisted Onsite Data Collection and Deep Learning Enabled Robotic Vision System for Identification of Cracks on Diverse Surfaces
This paper discusses the architecture of ROAD (Robotics-Assisted Onsite Data Collection System) as a means of automatically detecting cracks and defects in road infrastructures. The paper looks into traditional methods of crack detection and their limitations and encourages the use of deep learning in crack detection. The paper also discusses the effectiveness of multiple deep learning algorithms in detecting cracks on roads and concludes that Xception has the best performance with an accuracy over 90% and mean square error of 0.03. More generally the paper claims that deep learning algorithm trained in good datasets outperform the traditional methods. The reason why the authors push for ROAD is due to the lack of automation when it comes to traditionally detecting cracks in roads and therefore introduces ROAD (Robotics-Assisted Onsite Data Collection System), which integrates robotic vision, deep learning, and Building Information Modeling (BIM) for real-time crack detection and structural assessment.
Novel pavement crack detection sensor using coordinated mobile robots
The paper proposes the design of an integrated unmanned ground vehicle (UGV) and drone system for real-time road crack detection and pavement monitoring. A drone conducts an initial survey using image analysis to locate potential cracks, while the UGV follows a computed path for detailed inspection using thermal and depth cameras. The collected data is processed using MATLAB and CrackIT, enhanced by a tailored image processing pipeline for improved accuracy and recall. A crowd-sourced crack database was developed to train and validate the system. Webots software was used for simulation, demonstrating the system’s effectiveness in structural health monitoring. The proposed system offers high mobility, precision, and efficiency, making it suitable for smart city applications.
Pixel-Wise Crack Detection Using Deep Local Pattern Predictor for Robot Application
This study introduces a novel crack detection method using a Convolutional Neural Network (CNN)-based Local Pattern Predictor (LPP). Unlike traditional methods that classify patches, this approach evaluates each pixel’s probability of belonging to a crack based on its local context. The proposed seven-layer CNN extracts spatial patterns, making the method robust to noise, lighting variations, and image degradation. Experiments using real-world bridge crack images demonstrate superior accuracy over existing methods (STRUM and block-wise CNN). The study also explores optimized sampling techniques and Fisher criterion-based training to enhance performance when datasets are limited. The method shows potential for real-time crack detection in robotic vision applications.
Development of AI- and Robotics-Assisted Automated Pavement-Crack-Evaluation System
The paper presents AMSEL, a semi-automated robotic platform designed to inspect pavement cracks in real-time using a deep learning model called RCDNet. The system uses both manual and automated navigation to collect data indoors and outdoors, with RCDNet detecting cracks based on image analysis. Despite some limitations, such as difficulty detecting cracks smaller than 1 mm and issues with lighting and shadow interference, the system provides an efficient alternative to manual inspections. Future improvements include integrating non-destructive testing (NDE) sensors, expanding the use of visual sensors for faster coverage, and developing deep learning models that can fuse data from multiple sources for more comprehensive defect detection.
Robotic surface exploration with vision and tactile sensing for cracks detection and characterization:
The paper Robotic Surface Exploration with Vision and Tactile Sensing for Cracks Detection and Characterization suggests a hybrid approach to crack detection by complementing vision-based detection with tactile sensing. The system first employs a camera and object detection algorithm to identify potential cracks and generate a graph model of their structure. A minimum spanning tree algorithm then plans an effective exploration path for a robotic manipulator that reduces redundant movements.
To improve the accuracy of detection, a fiber-optic tactile sensor mounted on the manipulator verifies the presence of cracks, removing false positives from lighting or surface textures. Once verified, the system performs an in-depth characterization of the cracks, pulling out significant attributes such as length, width, orientation, and branching patterns. The two-sensing modality yields more precise measurements than traditional vision-only methods.
Experimental validation demonstrates that this integrated approach significantly enhances detection accuracy while reducing operating costs. By optimizing motion planning and reducing reliance on full-surface scanning, the system offers a more efficient and less expensive method of automated infrastructure inspection and maintenance.
Work Records
Week 1
| Name | Hours | Work |
|---|---|---|
| Isaak Christou | 8 | Made the wiki page, problem statement |
| Luca van der Wijngaart | 1.5 hours | Group meeting, first start to the Approach section |
Week 2
Week 2
Week 3
Week 3
Week 4
Week 4
Week 5
Week 5
Week 6
Week 6
Week 7
Week 7
Week 8
Week 8