PRE2024 3 Group10

The OneDrive can be accessed via https://tuenl-my.sharepoint.com/:f:/g/personal/l_kirkels_student_tue_nl/Ev3UBVOtfktApfNvxowBf-YBFpdR1uAtqIWQzg_Px7CosQ?e=URlCi2

Every week we have divided the task between chairman, minute taker and a wiki updater. The wiki updater updates the OneDrive at the end of the week to the Wiki page.

Group members

Member	Student number	Program
Loes Kirkels	1898477	Applied Physics
Joost Schuurmans	1893297	Applied Physics
Jamin van Amelsvoort	1829998	Applied Physics
Tom Weegels	1883410	Applied Physics
Norah Bouma	1902121	Applied Physics

Problem statement and Objectives

Beaches serve as vital ecological, recreational, and economic assets, yet they are increasingly compromised by plastic pollution. The accumulation of plastic waste on shorelines, estimated at 20-80 kg per kilometer, poses significant environmental challenges, affecting marine ecosystems and reducing the aesthetic and economic value of these coastal areas. If not addressed, plastic pollution can lead to long-term ecological degradation and economic setbacks for coastal communities reliant on tourism and fisheries.

Efforts to mitigate this issue have primarily relied on organized beach cleanup initiatives, such as the Boskalis Beach Cleanup Tour, which mobilizes volunteers to remove plastic waste along the Dutch coastline. These initiatives also contribute valuable data concerning the quantity, type, and distribution of plastic waste. However, as these efforts predominantly take place during the summer months under favorable weather conditions, the data collected presents a skewed representation of the extent of plastic pollution. Seasonal variations, storm-driven plastic accumulation, and tidal influences are not adequately captured, leading to an incomplete understanding of the pollution problem.

To develop a comprehensive strategy for addressing plastic pollution, it is essential to gather data throughout the year, including in adverse weather conditions. This research focuses on Castricum Beach in the Netherlands, investigating how autonomous robotic solutions can facilitate year-round data collection and plastic removal. The study will examine the current state of pollution at Castricum Beach, evaluate AI models for plastic detection, and determine the most suitable sensors and imaging technologies. Additionally, the research will assess the specific needs of stakeholders involved in marine conservation and policymaking to ensure that the proposed solutions align with practical and regulatory requirements. The primary focus will be on using robotics to help Stichting de Noordzee with cleanups in bad wheater conditions, as it plays a leading role in marine conservation efforts in the Netherlands.

Users and their needs

The primary user is Stichting de Noordzee which is an organization that specializes in cleaning the North sea in the Netherlands, including the beaches. They need a lot of data, over all periods, to get the full picture of the types of plastic, where exactly they are, in which time periods, etc. We have been in contact with them via mail, and are conducting an interview on the 10th of March. In this interview we want to get to know what they are currently doing, and what exactly they need, so maybe focus on a specific type of plastic, a specific weather condition, a specific period of time, etc. After this interview even deeper research will be done in specific elements of a solution of plastic data gathering in bad weather conditions. The interview can be found under 'Interview Stichting the Noordzee'

Primary users:

1.Stichting de Noordzee (The North Sea Foundation)

Stichting de Noordzee is a key environmental organization focused on preserving the North Sea and its coastlines. They are responsible for beach cleanup efforts, including large-scale initiatives such as the Boskalis Beach Cleanup Tour. To effectively combat plastic pollution, they require comprehensive and year-round data on plastic waste, including:

• The types of plastic found on beaches.
• The precise locations where plastic accumulates.
• Seasonal variations in pollution levels.
• The impact of weather conditions (e.g., storms, tides) on plastic accumulation.

Their primary need is a solution that enables reliable plastic detection and removal, even in challenging weather conditions when manual cleanup is difficult.

2. Beach Cleanup Volunteers and Teams

Volunteers play a significant role in organized cleanup efforts, but their work is limited by time, weather, and accessibility. A robotic system capable of autonomously detecting and removing plastic waste would assist these groups by:

• Reducing the manual workload required for beach cleanups.
• Operating in harsh weather conditions where human volunteers may be unable to work.
• Providing real-time data on plastic waste distribution to enhance planning and efficiency.

Their needs include a user-friendly and cost-effective solution that can be integrated into existing cleanup efforts without requiring extensive technical knowledge.

Secondary users:

1. Researchers and Scientists

Academic and environmental researchers studying plastic pollution, marine ecosystems, and AI-based detection technologies could benefit from the data collected by the autonomous system. Their key requirements include:

• Access to structured datasets on plastic pollution trends over time.
• Insights into how weather and tidal conditions influence plastic accumulation.
• Information on the effectiveness of AI models in detecting different types of plastic.

This research could contribute to a better understanding of plastic pollution patterns and inform future scientific studies.

2. Engineers and Technology Developers

Robotics engineers, AI specialists, and environmental technology developers could leverage the findings of this study to improve existing robotic cleanup solutions or create new ones. Their needs include:

• Detailed technical documentation on sensor performance and detection accuracy.
• Data on the environmental challenges faced by autonomous robots operating in beach conditions.
• Feedback on AI model effectiveness in real-world scenarios.

This information would help refine robotic designs, enhance AI detection models, and optimize system reliability.

3. Policymakers and Government Agencies

Local, regional, and national government agencies responsible for environmental protection and waste management could use the research findings to support policy decisions. Their key requirements include:

• Reliable data on plastic pollution levels to justify funding cleanup initiatives.
• Evidence-based recommendations for integrating autonomous solutions into environmental policies.
• Compliance with national safety and environmental regulations for deploying robotic systems in public areas.

The success of an autonomous robotic cleanup system could influence future regulations on beach maintenance and contribute to the development of new standards for plastic waste management.

State of the art Literature research

To ensure the effectiveness of autonomous robotic solutions in beach cleanup operations, this study will explore several key technological components. An initial assessment of Castricum Beach will be conducted to evaluate the extent and distribution of plastic waste across different seasons. This will involve field studies to document pollution levels, identify sources of plastic waste, and analyze seasonal trends influenced by tidal activity and storm surges.

The study will also investigate AI-based plastic detection models to determine the most suitable approach for identifying and categorizing plastic debris in various environmental conditions. Deep learning models such as YOLO (You Only Look Once), Mask R-CNN (Region-based Convolutional Neural Networks), and U-net will be examined for their effectiveness in distinguishing plastic waste from natural materials. Consideration will be given to the trade-offs between accuracy, processing speed, and computational efficiency.

In terms of hardware, different imaging technologies will be evaluated for their suitability in detecting plastic waste under varying lighting and weather conditions. RGB cameras, thermal imaging, and hyperspectral sensors will be compared to assess their effectiveness in identifying plastic debris in wet, low-visibility, and high-reflection environments. LiDAR and depth-sensing technology will also be considered to improve object detection and classification accuracy.

Additionally, the study will explore the engineering challenges associated with deploying autonomous robots in beach environments, focussing on Castricum. These include mobility on wind, resistance to harsh weather conditions, and energy efficiency. (Various power sources, including solar panels and high-capacity batteries, will be analyzed to determine optimal solutions for sustained operation in remote coastal areas.)

Drones in bad weather conditions

Paper 1: A.B. Bello et al. Fixed-Wing UAV Flight Operation under Harsh Weather Conditions: A Case Study in Livingston Island Glaciers, Antarctica. Drones, 6(12), 384; https://doi.org/10.3390/drones6120384, 2022

Drone used: fixed-wing RPAS, in particular, a Trimble UX5 UAV with electric pusher propeller by brushless 700 W motor, chosen for its ability to fly long distances and reach inaccessible areas.

Various hypotheses are proposed and tested, based on the main variables that can affect the flight operations (aircraft stability in flight under gusty winds, behavior of the drone’s materials at low temperatures, battery life, camera configuration to reduce the effects of albedo, etc.). Weather conditions of measurement:

Important variables for drone performance: air temperature, relative humidity, atmospheric pressure, precipitation, wind speed and direction, insolation (zonnestraling), and albedo (lichtweerkaatsingsvermogen).

The general rule of thumb for flying drones is that the wind speed should be no more than two-thirds of the maximum speed of the drone. It has also to be borne in mind that the higher the wind speed, the greater the consumption of battery power.

Hypotheses:

1. withstand lateral winds of up to 50 km/h and gusty winds of up to 15 km/h with a cruise speed of 80 km/h.

1. battery life decreases less than 30% due to cold weather

1. The main materials were high-density polyethylene foam and carbon fiber frame structure and composite elements. Accessory elements, such as the shuttle, also had to be taken into account. None of the materials could severly be affected

1. Camera requirements (not in the scope of this research)

1. the aircraft could travel 60 km in 50 min, reserving 5 min for takeoff and another 5 min for landing.

You cannot fly drones in all weather conditions. à Not too windy or foggy. The reason is that, due to the low temperatures, the fog is formed by ice crystals, which cause Pitot tube obstruction and airframe gelation.

Results: The duration of the batteries was reduced by 30% due to cold conditions, and 5 min must be reserved for return and landing. Biggest problems are wind and rain. The plane was able to fly with constant winds of up to 35 km/h, withstanding gusts of 65 km/h. Because of cold temperatures, the battery live became considerably worse. This implied that flights had to be planned with a reserve of 30%.  In general, the materials were not affected by low temperatures. Due to the low temperatures at flight height there was sometimes icing of the aircraft wings, making them crash.

Paper 2: Gao, M., Hugenholtz, C.H., Fox, T.A. et al. Weather constraints on global drone flyability. Sci Rep 11, 12092 (2021). https://doi.org/10.1038/s41598-021-91325-w  

The study analyzes the impact of weather on drone operability using a 10-year dataset from the ERA5 reanalysis. Two drone classes were considered: Common Drones (CDs), which are inexpensive but have limited weather tolerance (0–40°C, max wind speed 36 km/h, no precipitation tolerance), and Weather-Resistant Drones (WRDs), which are more robust (-20–46°C, max wind 50 km/h, can operate in 50 mm/h rain).

Conclusion: DC’s are very sensitive to rain and seasonal temperature changes, as WRD’s are not really affected by it. Flyability of drones in Netherland’s coastal region should not be a problem with WRD’s.  

Situation in Castricum

In the winter months it can get up to about -6 degrees celsius in the night, but on average the lowest will be about 2 degrees during the whole day.

During the winter months, the wind speed can go up to about 35 knots or more, which is about 65 km/h or more, so that is very fast.. On average though the wind speed will be about 30-40 km/h.

All the plastic found during a cleanup on the Canticum beach is laid out here. The small pebbles are plastic pellets which are very small in size, about ten times maybe even more smaller than a cigarette bud, so very tiny. The rest are mainly broken pieces of plastic, cigarette buds, or other various types of plastic.

Cameras and Sensors

For robots with the goal to detect plastic on beaches, the following cameras and sensors can be used. They each have advantages or disadvantages for usage in bad weather conditions.

• RGB cameras
  • Capture standard visual images and can be paired with AI-based image processing algorithms to recognize plastic waste. They identify plastic based on color, shape, and texture differences compared on the sand.
  • Struggles in low lights when visibility is reduced, because the cameras rely on visual images.

• Multispectral and hyperspectral cameras
  • Capture specific wavelengths of light beyond the visible spectrum. Due to this plastic can be distinguished from organic matter.
  • Perform well under clouds but may be affected by wet weather, as moisture can alter spectral readings.

• Thermal infrared cameras
  • Plastic absorbs and retains heat differently than natural materials. Thermal infrared cameras can notify these differences, thus differentiating the plastic.
  • Effective in cold and cloudy conditions, because plastic does not retain heat from its surroundings.  

Sensors:

• LiDAR (Light Detection and Ranging)
  • Use laser pulses to map the terrain and detect objects in 3D. These type of sensors can be used by a robot travelling on the ground.  
  • Not effective in heavy rain or snow due to light scattering of the laser pulses.

• Near-Infrared (NIR)  
  • Analyzes the reflectance properties of materials, being able to differentiate plastic.  
  • Can be affected by wet surfaces

• Ultraviolet (UV) Sensors
  • Take advantage of the fluorescence exhibited by some plastics under ultraviolet light.
• Laser-Induced Fluorescence (LIF) Sensors
  • LIF uses a laser to excite plastic materials, causing them to emit characteristic fluorescence. This method is effective even under cloudy conditions since the laser provides its own illumination.
  • Highly effective in low-light and cloudy conditions, as it uses an active laser source, making it one of the most reliable methods for detecting plastic in adverse weather.

Thus, for a robot that is used to detect plastics, it is best to use a thermal infrared camera with a sensor using Laser-Induced Fluorescence. With these combination, the camera contains images using heat, and observes the plastic using the fluorescence characteristics of plastic. Both of these are minimally affected by bad weather.

AI plastic detection

(https://uu.diva-portal.org/smash/get/diva2:1899954/FULLTEXT01.pdf)

This paper investigates multiple AI learning models on the TACO dataset, which is a database consisting of all kinds of different plastics. They look at three different CNN (Convolutional Neural Network) algorithms, Mask Region-based Convolutional Neural Network (Mask R-CNN), You Only Look Once (YOLOv7), and U-Net, they then use two different machine learning models on these, k-Nearest Neighbors and Logistic Regression, were trained and evaluated. How this roughly works is that you have a dataset, in this case the TACO database consisting of images of all kinds of plastics. The CNN algorithms break down these pictures into very small pieces and look for similarities. These are mostly used for breaking down these pictures which are then classified with a different machine learning algorithm. Now you have classified a lot of pictures and if you use this trained model on a totally new picture it should be able to classify this new picture into one of the classes created by the machine learning algorithms. Their findings are that firstly the TACO dataset is quite imbalanced, favouring some types of plastic a lot more than others, a lot of pictures of cigarettes and plastic bags/wrappers, while straws, styrofoam and aluminium foil are very limiting. The confusion matrices of the trained models with Mask R-CNN and YOLOv7 can be seen below. The ones in the diagonal row from top left to bottom right, are correctly classified objects of a specific class.

An example of how the model detects plastic can be seen below, this is the YOLOv7 trained model. Boxes are roughly put around the object, a class is assigned to them with some specific probability of it being correct based on the data it has been trained on. So, if we figure out what exactly we want to detect it can be solely trained on this specific aspect to be very good at this one thing. The k-NN trained model had a total accuracy of 64%, while being very good in detecting bottles (82%), and plastic film (74%). The logistic regression model had a overall accuracy of 73%. So the logistic regression model is much more balanced than the k-NN trained model

They state that the current detection with these trained models is okay, but not far from perfect. More types of different litter categories are needed with a lot more data for some categories, because the data is skewed right now.

Interview Stichting the Noordzee with Tako Popma (script)

Thank you for taking the time to speak with us today, Tako. This conversation will take place in Dutch via Microsoft Teams, but we have prepared our questions in English.

We are researching pollution in Dutch rivers and the sea and exploring how robotics could help reduce this pollution. We read about Stichting De Noordzee’s Beach Cleanup Tour and were particularly interested in the challenges you face during these cleanups, especially in difficult weather conditions.

We also learned that Stichting De Noordzee has previously worked with cleanup robots during the Beach Cleanup Tour. We would love to hear about your experiences with this project and discuss how robotics could further support cleanup efforts, particularly in locations like Castricum, where harsh weather can make manual cleanup more difficult.

Beach Cleanup Challenges

• Can you describe the main challenges that your team faces during beach cleanups, especially in harsh weather conditions like strong winds, heavy rain, or cold temperatures?
• How do these conditions affect volunteer participation and the overall efficiency of cleanups?
• Are there specific types of pollution that are harder to remove manually in these conditions?

Experiences with Cleanup Robots

• We read that Stichting De Noordzee has previously worked with cleanup robots during the Beach Cleanup Tour in your mail. Can you tell us more about this project?
• What kind of robotic system was used, and how did it function?
• What were the main advantages and disadvantages of using this technology?
• Were there any unexpected challenges or technical difficulties in using robots for cleanup operations?

Robotics and Future Cleanup Operations

• Based on your experience, do you think robots could play a larger role in future beach cleanups? Why or why not?
• Are there tasks during cleanups that you think robots could take over or assist with, such as detecting, collecting, or sorting waste?
• Do you see any risks or limitations in using robotics for environmental projects?
• How could robotics help improve cleanup efficiency in places like Castricum, where bad weather can be an obstacle?

Implementing Robotics in Beach Cleanups

• From your experience, what features would a robotic cleanup system need to function effectively in a Dutch beach environment?
• What kind of pollution or waste do you think would be the most important for a robotic system to focus on?
• Could robotics help make cleanups more consistent, especially in months when volunteers are less available due to bad weather?
• Are there specific areas or locations along the Dutch coast where robotics could be particularly beneficial?

Collaboration and Future Perspectives

• Would Stichting De Noordzee be open to testing robotic solutions for beach cleanups in the future?
• How do you see the future of beach cleanups evolving with technology?
• What do you think is the most important aspect to keep in mind when integrating robotics into environmental projects?

Final Thoughts

• If you could introduce one major improvement to the way beach cleanups are currently done, what would it be?
• How can people best support Stichting De Noordzee’s efforts in keeping the North Sea clean?

Thank you for sharing your expertise with us, Tako. Your insights will be incredibly valuable for our research on how robotics can support and improve cleanup efforts in challenging environments. We appreciate your time!

Planning

	Approach	Milestones and deliverables
Week 1	-Make planning with milestones and deliverables (everyone) -Literature study (5 articles per person), summarized on the wiki (state-of-the-art research) -Make problem statement and objectives (Jamin) -Write down users and what they require (Joost)	Problem statements and objectives Plan (subject, objectives, users)
Week 2	-Make a state-of-the-art from the summarized articles (Norah) -Think of the requirement and goals the robot should have (Loes&Tom) -Do further research (Everyone) -Try to look for organizations and try to contact them	State-of-the-art Concrete ideas for a concept
Week 3	-Write down the research in the wiki (Everyone) -Investigate the specifics of the robot (Jamin&Loes) -Write the user study in the wiki (Tom) -Write down the process of approach in the wiki (Norah)	Finished research Have a rough overview Contact with Stichting de Noordzee
Week 4	-Design concept -Write the research and results in the wiki (Joost) -Start designing the model of the robot (Loes&Norah)	Research and results in the wiki Interview Stichting the Noordzee
Week 5	-Finish the model of the robot (Loes&Norah) -Finish the wiki (Joost&Jamin&Tom)	Finalized model Finalization of research, everything written down in the wiki
Week 6	-Make the presentation -Room for things that still need to be done	Presentation
Week 7	Preparation for final presentation

Weekly updates

	Loes	Joost	Jamin	Tom	Norah	General
Week 1	Research papers (6h)	Research papers (4h)				Monday morning lecture + discussion (2h), Wednesday morning working session (4h)
Week 2	More specific research (4h), Contacting different organizations (2h)					Monday morning meeting (1h), monday afternoon meeting (4h), contact users (4h), generalize and research the subject (2h)
Week 3	Contacting stichting the Noordzee further (1h), Preparing/ setting the interview with Stichting the Noordzee (2h), Changing/Expand the wiki (Users and problemstatement) (1h) Setting up the interview script (1h)	Research drones in bad weather conditions (4h)				Monday morning meeting(1h), monday afternoon meeting (4h), research/contact organizations (1h)