PRE2020 4 Group1

Project Title t.b.d.

<div#bodyContent sup {

   font-size: smaller;
   vertical-align: baseline;
   position: relative;
   bottom: 0.33em;

}

1. bodyContent sub {

   font-size: smaller;
   vertical-align: baseline;
   position: relative;
   bottom: -0.25em;;

}>

Team members

Members	Student ID	Faculty	E-mail
Kashan Alidjan	1224924	Electrical Engineering	k.m.s.alidjan@student.tue.nl
Sijt Hooghwinkel	1228761	Automotive	s.j.c.hooghwinkel@student.tue.nl
Damaris Jongbloed	1241057	Computer Science	d.a.jongbloed@student.tue.nl
Emma van Oppen	0963999	Computer Science	e.y.v.oppen@student.tue.nl
Laura Verbeek	1428063	Biomedical Engineering	l.h.e.verbeek@student.tue.nl

Introduction

In 2020 the Dutch government spent 8.9 billion euros on roads, railways and waterways. In the Netherlands, there are 125,575 kilometres of roads, which amounts to 27000 euros per km. People new to the Netherlands often claim how well-maintained our roads are, and the Netherlands ranks second in road quality worldwide. Road quality is valued highly in the Netherlands due to a large number of cyclists. Minor damage will not be noticeable in a large car, but small cracks can lead to dangerous situations on a bicycle. To keep the roads in such great condition, a lot or work is done. Municipalities, provinces and the Ministry of Infrastructure and Water Management work round the clock to repair damage, but they all use a different system. Most damage is detected by periodic checks, road workers and civilian complaints. To improve the efficiency and cost of these detections, this project intends to present a solution that passively detects damage and stores this in a central database, to be inspected by the responsible parties.

In order to investigate the need of such a device, a survey was sent out to municipalities, asking them if they see a need for such a detection device. 34 out of 52 replies indicated that a global detection system could be an improvement (see appendix), and more in depth feedback was provided. For example, most municipalities use the CROW mechanism to decide priority of repair. The municipality Bladel says: "A guideline has been developed for road management that serves as a tool to arrive at the most optimal long-term planning: that is the actual translation of the quality level into concrete maintenance measures. This road management system is laid down in publications 146 and 147 of the CROW. This is the most important standard work for road management in the Netherlands. This system is used by most municipalities, provinces and water boards and is often seen as a guideline in case law."

Furthermore, civilian complaints were done in different interfaces, such as BuitenBeterApp, Fixi or the Meld & Herstel app. Reimerswaal said: "FIXI reports from residents are immediately passed on as an order for repair to a number of house contractors who work for us on a contract. This if the report is serious enough, which may be apparent from the enclosed photo or recording by our on-site supervisor." Ijsstelstein added: "A national reporting system that all municipalities can use is an improvement. There are so many now, this causes too much noise on the reports from the public space, this delays and causes dissatisfaction for the residents." The main concerns indicated in the survey were that different municipalities have different budgets for repair (as indicated by Kampen and Ijsselstein), there could be too many or too few detections (due to inaccuracies) and all damages require manual inspection anyway.

Objectives

Following the survey, requirements were gathered to lead this project to success.

1. The product needs to save money. Without this, there is no interest from municipalities, to whom the product is aimed. If the product is sufficiently cheap, real-world implementation is easier.
2. The damage needs to be reported to the right road authority. The current system for road repair is individual, and it was indicated this is an important feature to assign repair.
   1. Furthermore, accurate location data of the damage should be reported.
3. Only damages should be reported, for example, speedbumps should be ignored. However, if the damage is not severe enough, and does not need to be repaired, it should not be reported either.
4. The CROW framework should be supported. Since most municipalities use this framework, it makes sense to provide an implementation in the database.
   1. Accordingly, priorities can be assigned to reports.
5. Some damages will only be noticed with the eye, so these should be reported manually.
6. Additionally, the product will provide more profit when implemented on a large scale. Therefore, it should be easy to implement and use.

Dit deel mist nog motivatie:

The main objective of this project is to develop a device that measures road surface quality by detecting irregularities in the pavement, such as potholes and cracks, that are encountered while driving around. This device can be placed onto a vehicle and will collect data on these irregularities using several different sensors, such as an accelerometer and a GPS module.

Through the use of crowdsourcing, the collected data, along with GPS locations, can be used to visualize the locations of potholes, cracks and other damage to the road’s surface, thus creating a mapping of the overall road quality. This data can be used to quickly assess where the damage is the most severe and which roads are in need of repair. As such, road maintenance can be planned more immediately when problem areas are detected, which can contribute to an increase in road safety.

1. Collect data on potential locations of potholes, cracks and other road surface damage.
2. Visualize the locations of detected road damage by overlaying the collected data on a map.
3. Enable quick assessment of where the damage is most severe and repair is necessary.
4. The device must be cost effective to enable widespread usage.

USE analysis

Users

The public roads in The Netherlands are not maintained by one organization. The management is mainly shared between Rijkswaterstaat (highways), provinces and local municipalities. The two latter are responsible for the “N” roads, non-highway roads.

People that have the device installed on their vehicle can be considered as passive users as they do not have to interact directly with the device if they wish. They will need to monitor the physical state of the device and have the device replaced/repaired if needed. The users can also actively interact with the App to see their contribution and edit wrongful detections of the system (false positives) or add locations where the system has not detected anything (false negatives).

Responsible organization for different roads, Leusden, The Netherlands

Society

Societal stakeholders are all road users.

Enterprise

If a road repair needs to be done a contractor will be used to perform this repair.

Depending on the means of installation and technical knowledge that is required to perform installation (TBD!), this may need to be done by a garage. Several garages could be selected to have an inventory of the devices to install on customers cars during their general inspection (APK) or with a service.

Car insurance companies are also stakeholders in this project. Car owners could make use of their insurance policy to claim for damages on their vehicle because of bad roads. Better quality roads would mean insurance companies would have to compensate fewer people.

State of the art / Research

Similar research has already been conducted. The recognition of potholes is done in various ways. The main designs made are as follows.

1. Implementation of Smartphones
2. Accelerometer
3. Profilometer
4. Camera/scanner
5. Vehicles (level sensor, acc)
6. Deep learning AI

A commonly used device is peoples smartphones, as those contain rather accurate accelerometers and have an integrated GPS already provided. Moreover, it can transfer data easily, as that is the main feature of smartphones. Therefore, reading out data from a smartphone is considered efficient, as it has all the sensors and data transfer applications already present for the user. However, the main drawback to using smartphones is that most of these applications require the smartphone to be held in a specific position. For example on the windshield or mounted to the dashboard. Another important drawback is that the software introduced poses a lot of strains to the smartphone. The constant access to GPS data and the constant use of the high-precision accelerometer requires a lot of the battery. Next to this, the app needs to be turned on. An improvement would be to have the app run in the background and only access GPS data when a pothole is measured. Other applications also include making a whole system that collects the data, however, this system collects a lot of data, which might be unnecessary to the agencies that oversee reparation.

Besides the application of a smartphone, the use of separate sensors is broadly considered as well. Here, the most important sensors are the accelerometer, the profilometer, a camera/scanner, lasers/reflections and SDCs with implemented sensors. The accelerometer measures acceleration in different directions (xyz). The profilometer measures the roughness of a surface. This is extremely useful for identifying road quality using the IRI. Yet the data communication between the road agencies and the developers seems to be in the development phase.

The most advanced technique of locating potholes is done using a camera and deep learning AI. Here, the camera is aimed at the road and will visually assess the road for potholes.

The main technique consists of the following: The pothole is detected (accelerometer, profilometer, camera, etc.), the location is attached to it (GPS) and the data is collected (system, server).

Plan

Approach & Milestones

To meet our goals, we subdivided our research into 12 milestones. For each milestone, a short description is given to clarify what needs to be done to reach the milestone.

1. Find a subject: to do the project, we need to determine the subject that we will focus on. For this, we should think about what type of research we want to conduct (e.g. literature analysis, experimenting, designing, building a prototype…). Other things to keep in mind are:
   1. What are the strengths and weaknesses of the group?
   2. Which projects are possible to do within 8 weeks?
2. Literature study: the second milestone is to gain knowledge about the possibilities already used for our problem. In this literature study, we will look at the State-of-the-Art, and what the pros and cons are of these solutions to the problem. This knowledge can be applied to improve our own research.
3. Survey: to find out about the needs of the users, we will do a survey. It will help us define the problem.
4. Conceptual design: in the fourth step, we make multiple designs for the problem. For each design we will look at the pros and cons to find the best design. We can choose the best design or combine the strengths of multiple designs to form a new concept.
5. Functional description: once we know what the concept will look like, we need to figure out which components are needed to build the main parts of the design, and we have to look up which components/materials are available in the store or online. For example: how is the product supplied with power? A battery? A cable? Furthermore, we should start working on the software.
6. Final design concept: using the knowledge about the materials found in (5), functional description, we should revise the conceptual design and make some adjustments if needed. The conceptual design will be refined to make the final design concept.
7. Detailing: the next step is detailing, which means that we should look at the specifications of all the components. Using the same example as in (5), functional description, which battery is sufficient to keep the product working for an X amount of time. In this step it is also important to look at the dimensions of all the components to fit them all in the casing.
8. Mock-up: making a mock-up of the concept using cheap materials and/or a CAD design.
9. Software: once the functional description is made, a start should be made on the software. The software should be finished before the realization (10).
10. Realization: once we are satisfied with the prototype, we should order the components and start building the concept (make a BOM). First all components should be tested independently before putting the concept together. Hardware and software are combined.
11. Testing: the prototype should be tested. (Minor) adjustments can be made in the software and/or hardware to refine the product.
12. Results and evaluation: evaluation of the results of the test will give insight in the quality of the product. Possible areas of improvements are mentioned in the evaluation, as well as the strengths of the design.
13. Final presentation and discussion: finalize the wiki page and prepare for the final presentation.

Deliverables

The deliverables for this project consist of the following items:

1. The physical prototype
2. An app that collects the data output of the product
3. A demonstration of the product, shown during the video presentation
4. The wiki page containing main topics, such as:
   1. Problem statement
   2. USE aspects
   3. Product
   4. Planning
5. A video presentation regarding the product, process and most important findings.

Planning

Week	Milestone	Milestone name	Task	Responsible	Deadline
1	1	Find a subject	Brainstorm	All	25-4
	2	Literature study		All	25-4
2	2	Literature study	Finish literature	Emma, Laura	29-4
	3	Conceptual design	Sketch app	Emma, Sijt	2-5
	3	Conceptual design	Sketch physical design	Kashan, Laura	2-5
	3	Conceptual design	High level interaction (USE diagram)	Damaris	2-5
	1	Find a subject	Problem definition rewrite	Emma, Laura	2-5
	1	Find a subject	Problem definition research	All	2-5
3	4	Functional description		All (tbd)	6-5
	5	Final design concept		All (tbd)	9-5
4	6	Detailing		All (tbd)	13-5
	8	Software		Emma, Sijt	16-5
			Order parts	Tbd	16-5
5	7	Mock-up		Kashan, Laura, Damaris	23-5
	8	Software		Emma, Sijt	23-5
	10	Testing	Testplan, perform tests	All	23-5
6	9	Realization	Perfect product	Kashan, Laura, Damaris	30-5
7	11	Results and evaluation		All	6-6
	12	Presentation and discussion		All	6-6
8			Overtime	All	13-6

Product Research

State of the Art

Sensors

In order to detect road anomalies several sensors could be used. In this section a concise advantage/disadvantage is given per sensor.

There are two systems that could be used, Arduino and standalone sensors. Arduino is preferred here because of component compatibility and being budget friendly. Standalone sensors such as the "Inertia ProMove 3D Motion Tracking" as used in would become cumbersome to integrate.

The following sensors are available to be used for Arduino:

• Ultrasound
  • HC-SR04
• Accelerometer + Gyroscope
  • MPU-6050 (Comes in pair)
• IR Sensor
• Camera
  • OV7670 CMOS
• GPS
  • GY-NEO6MV2
  • ATGM336H

Advantages:

• Ultrasound
  • Measuring range from 2cm to 2m
  • Measurement interval of 30 microseconds
• Accelerometer + Gyroscope
  • Comes integrated as a pair
  • Can be placed inside the vehicle
• IR Sensor
  • -
• Camera
  • Could detect smaller road anomalities
  • Could save a reference picture

Disadvantages:

• Ultrasound
  • Needs to be perpendicular to the road
    • Will be exposed outside of the vehicle
  • Has a certain inaccuracy
    • during a test at 10cm distance values varied between 9.35cm and 10.39cm
• Accelerometer + Gyroscope
  • niks
• IR Sensor
  • -
• Camera
  • Heavily dependent on lighting conditions
  • Low resolution and FPS available for Arduino
    • 640x480, 30fps
      • With 30fps at 100km/h (28m/s), 1 frame is captured every meter
  • Requires complex and computing intensive post processing to implement detection
  • Requires good vibration dampening

A smartphone already has a GPS, camera, gyroscope and accelerometer installed. The following section will go more in-depth on using a smartphone.

Smartphone

Smartphone positioning

Nearly every smartphone in existence nowadays contains a global positioning system (GPS). The GPS in such smartphones has an accuracy of 4.9 meters on average . In the case of Android phones, two different providers can be used to determine the phone’s location: the GPS location provider and the network location provider. It is advised to use both when you want to continually update the phone’s location, as GPS is inaccurate when the direct line between the satellites and the phone is interrupted, e.g. when the phone is located indoors, in a tunnel or in an urban area with many tall buildings, and the network provider is inaccurate when network connectivity is poor. The data of both providers can be compared to obtain an accurate position . When creating a smartphone application in Android Studio that needs constant updates on the user’s position, a LocationListener can be created that will continuously monitor changes in the data received from the GPS and network location providers. When a change is measured, a Location object is returned. This object has many attributes that can be useful for measuring road quality. Aside from the obvious longitude and latitude, it also has measured speed, bearing and a timestamp . Most inbuilt smartphone gps sensors have a maximum update frequency of 1 Hz, which is once per second. This is a relatively low frequency, and may result in inaccurate measurements, specifically when travelling at high speeds. For example, when a car is travelling at 100 km/h, this equals about 27.8 m/s, which gives a very large radius for where a detected pothole could be located. For high update frequency, it is important that the application is running in the foreground. If a device is running Android 8.0 or higher, the frequency with which location data can be retrieved from the user’s device by an app that’s running in the background is limited to a couple times an hour in order to preserve the battery . This would result in data that is so inaccurate that it would be useless for this project’s intended purposes.

Motion sensors

How the axes are configured in an android phone

Most Android smartphones contain at least an accelerometer, and many also include a gyroscope. Both of these sensors are always hardware-based. Phones may also contain different sensors, both hardware- and software-based . However, these may differ greatly between different models, so the focus will be only on the accelerometer and the gyroscope. From a software perspective, these sensors all generate so-called SensorEvents. A SensorEvent object contains an accuracy, the type of sensor that generated the SensorEvent, a timestamp and the measured values themselves as an array of floats . These values are the most important data for the purposes of this project. Similar to the positioning data, changes in sensor data can be monitored with a SensorEventListener. Whenever a new SensorEvent is generated, this object is passed to the listener. The length and content of the values array depends on the type of sensor that produced the data. In the case of the accelerometer, this array contains three distinct values. Each of these values corresponds to an axis x, y or z. They are each calculated by acceleration - gravity * alpha, where gravity is 9.81 m/s², and alpha is calculated as t / (t + dT), where t is the low-pass filter’s time-constant and dT is the event delivery rate. The gyroscope sensor also measures three different values; the angular speed around each axis for axes x, y and z. It is important to explain how the coordinate system works for Android smartphones. It is defined relative to the screen when it is in its default (upright) position. The x-axis is horizontal and points right, the y-axis is vertical and points upwards, and the z-axis is perpendicular to the x-axis. The coordinate system is always configured in this way, and does not change with the orientation of the smartphone.

The sampling rate for any accelerometer depends on the device itself, of course. While the maximum sampling rate that can be obtained with a smartphone’s accelerometer is 200 measurements per second, this is not the case for every smartphone. However, 100 samples per second can be guaranteed in almost all smartphones . According to another paper, where the sensors of six different phones were tested, the average sampling rate of the gyroscope and the accelerometer were quite similar at around 200 samples per second . This should be sufficient to detect most damage on the road surface, especially when travelling at lower speeds.

Smartphone vs. built-in device

As seen above there are a few options available for the final product. In general the final product would come down to two options; solely using a smartphone or using an independent (built-in) device that houses all the sensors. Both options are viewed as viable but come with their own tradeoffs:

• Smartphone:
  • + No product cost
  • + Potential for fast upscaling is larger
  • + Data can be sent directly to App in real-time
  • - Requires user interaction
    • User must turn on and off the application
  • - Heavy battery usage
  • Uses GPS and active internet connection
  • - Potential sensor data issues for different model and make smartphone
  • - Smartphone must be mounted securely and not moved during operation

• Independent device:
  • + Uses the same components in all devices
  • + No interaction with user required
  • + Device can be placed in most optimal location (inside and outside of vehicle)
  • - Cost of components and potential installation costs
  • - Depending on design may need to be installed by a technician
  • - Data needs to be offloaded wirelessly
    • Could still require the use of a smartphone, but not as heavy use as only using a smartphone

Conceptual design

Device

The prototype will be made with the Arduino Uno as the core. It will contain three modules:

1. The MPU-6050 accelerometer+gyroscope module
2. The NEO6MV2 GPS module
3. The HC 05/06 Bluetooth module

A picture of the breadboard implementation is shown below.

• insert figure

Android application

The standard view of the map in the application (1)

The concept design for the application is quite straightforward. The idea is to provide the users with a single, simple environment that they can use to view and edit all available damage reports.

When the app is started up initially, the user will immediately get a view of the map (1). The map can be moved around and supports zooming in and out, and all points of damage in the database will be rendered on the map in the form of markers.

In order to find a location quickly, a search function is available. The search bar is located at the top of the screen in the app (2). Searching for a location can be done by typing in the postal code, the address or the coordinates of the desired location. After tapping either the return key on the keyboard or the search button located next to the search bar, the application will move the map to the specified location.

As mentioned earlier, the map shows markers for every location where a damage report was made, and this is the most important functionality of the app. Users can tap on each marker to show additional information (3). This will include at least the severity of the damage by some standard to be determined, and the exact location where the report was made, specified by the latitude and longitude.

The user can also choose to delete a marker, for instance, when the damage has already been repaired and the reports are outdated, or if the user knows that the marker refers to a false positive.

Additionally, each marker will receive a colour based on the measured severity of the damage. This will provide a nice overview on the map so that the user can immediately see where the damage is worst, without having to first check the supplied information.

The last functionality that the application offers is allowing the user to add markers to the map manually (4). This could be useful if they know of a location that requires repair, that isn't marked on the map yet. It might also be useful to allow users to specify a comment to accompany the marker that they are adding.

Searching for locations (2)

Manually add markers to the application (4)

A selected map marker shows a popup with its info (3)

Logbook

Week 1

Name	Student ID	Time spent	Tasks
Kashan Alidjan	1224924	7.5h	Intro (1h), Meeting (3h), Research (1.5h), Reading old projects (1h), Deliverables (0.5h), SoTA (0.5h)
Sijt Hooghwinkel	1228761	7h	Intro (1h), Meeting (3h), Reading old projects (1h), Users (2h)
Damaris Jongbloed	1241057	7.5h	Intro (1h), Meeting (3h), Research (1.5h), Problem statement (2h)
Emma van Oppen	0963999	8.5h	Intro (1h), Meeting (3h), Research (3h), Objectives(1h), Wiki(0.5h)
Laura Verbeek	1428063	9h	Intro (1h), Meeting (3h), Approach+milestones (2.5h), SotA (2.5h)

Week 2

Name	Student ID	Time spent	Tasks
Kashan Alidjan	1224924	9h	Meeting (3h), Design (1.5h), SotA summaries (2h), preparation (2h), wiki (0.5h)
Sijt Hooghwinkel	1228761	8h	Meeting (3h), Android studio (1h), researching OSM (1h), SotA summaries(3h)
Damaris Jongbloed	1241057	7h	Meeting 27-4 (1h), Meeting 29-4 (2h), Proofreading (0.5h), SotA (3h), Update planning (0.5h)
Emma van Oppen	0963999	14h	Meeting (3h), Research/SotA (9.5h), Wiki (0.5h), App Setup (1h)
Laura Verbeek	1428063	10.5h	Meeting (3h), reading the work of others (0.5h), SotA (5.5h), Design (1.5h)

Week 3

Name	Student ID	Time spent	Tasks
Kashan Alidjan	1224924	13h	Meeting 3-5 (3h), Meeting 5-5 (3h), Reactions (1h), Research sensors (3.5h), Writing code (1.5h), Testing design (1h)
Sijt Hooghwinkel	1228761	9h	Meeting 3-5 (3h), Meeting 5-5 (3h), Reactions (1h), Research sensors (2h)
Damaris Jongbloed	1241057	12h	Meeting 3-5 (3h), Meeting 5-5 (3h), Reactions (1.5h), Introduction (2h), References (1.5h), Requirements (1h)
Emma van Oppen	0963999	13h	Meeting 3-5 (3h), Meeting 5-5 (3h) + Travel (1.5h & €10), Reactions (0.5h), App (3h), Reseach (2h)
Laura Verbeek	1428063	9.5h	Meeting 3-5 (3h), Meeting 5-5 (3h), Rijkswaterstaat (0.5h), Reactions (0.5h), Update planning (1h), Draft questions (1.5h)

Week 4

Name	Student ID	Time spent	Tasks
Kashan Alidjan	1224924
Sijt Hooghwinkel	1228761
Damaris Jongbloed	1241057
Emma van Oppen	0963999
Laura Verbeek	1428063

Week 5

Name	Student ID	Time spent	Tasks
Kashan Alidjan	1224924
Sijt Hooghwinkel	1228761
Damaris Jongbloed	1241057
Emma van Oppen	0963999
Laura Verbeek	1428063

Week 6

Name	Student ID	Time spent	Tasks
Kashan Alidjan	1224924
Sijt Hooghwinkel	1228761
Damaris Jongbloed	1241057
Emma van Oppen	0963999
Laura Verbeek	1428063

Week 7

Name	Student ID	Time spent	Tasks
Kashan Alidjan	1224924
Sijt Hooghwinkel	1228761
Damaris Jongbloed	1241057
Emma van Oppen	0963999
Laura Verbeek	1428063

Week 8

Name	Student ID	Time spent	Tasks
Kashan Alidjan	1224924
Sijt Hooghwinkel	1228761
Damaris Jongbloed	1241057
Emma van Oppen	0963999
Laura Verbeek	1428063

Appendix

Municipal Questionnaire

The table below shows the results of the questionnaire sent out to several municipals in The Netherlands. The questions that was asked was: "Does the municipality see added benefit to using a global and continuous detection system for road quality?". In total 52 municipals reacted to the questionnaire.

Questionnaire answers from

Answer	Amount	General Comments
Positive	19	Doing more measurements is beneficial. Road user would not have to figure out how to contact municipal to report damages. A method used by every municipal would be positive.
Negative	18	Current system works properly. A sufficiently covering system is a utopia. Up until now there is no national system that benefits the municipals
Undecided	15	Would result in more alerts. Municipal would still have to check the damage but could save time. Only when saving money. Should support the CROW standards.

A large amount of municipals reacted either negatively or undecided. This is most likely because the question that was asked is very broad in terms of technical implementation. In most cases the contact person could not imagine how such a system would be implemented, hence being undecided.

State of the Art sources

Links of the State of the Art research:

References