PRE2024 3 Group11

https://docs.google.com/document/d/134X5otIGJQW5CHC_7DGSuY8oDns-jvgIoZ2rAsN69zg/edit?usp=sharing

Final presentation: https://docs.google.com/presentation/d/1UnbBNV1ydnzloF3R05HMhvIAnWFgMi6zT5n6MISVKuk/edit?usp=sharing

Name	Student ID	Department
Leander ten Hoedt	1663259	Computer Science and Engineering
Thijs van Berkel	1831917	Computer Science and Engineering
Pijke van Bohemen	1819755	Computer Science and Engineering
Mihaita Manolache	1457543	Computer Science and Engineering
Luca Borst	1669397	Electrical Engineering
Jort Arendse	1761285	Psycholgy and Technology

Introduction

Cities contain loads of birds who cannot find enough food to stay alive. Therefore, many people help these birds by supplying them with food like seeds or peanut butter. This helps most of these birds to get through the cold winter months. However, if you have ever looked outside at one of these food spots, you might have noticed that strong or large birds often hog these for themselves. This scares away small birds, who need food more than large birds. For example, in the Netherlands, a big subset of these strong birds are Magpies, who are quite aggressive. They will guard these feeding spots often and prevent tiny Sparrows from getting food.

To help these smaller birds, we want to create a feed-dispenser-type device. It will be able to detect what birds are around it and will close off access to the food if large birds are trying to eat. To help you as a user in finding a good spot and making sure that the container will not get empty. It will send notifications about which bird is eating food, it will also send the current food level.

State of the art research

Before conducting interviews to gain a better understanding of the problem and how the product should be designed state of the art research was conducted.

USE

Users

An automatic bird feeder can be made into a product suitable for multiple purposes and can be interesting for different users. Most birdfeeders are in gardens of people who simply like to watch birds from the comfort of their homes. However, they do not know when birds are feeding and thus when they can watch them. The automatic birdfeeder is ideal for this type of situation because it can send a message to the user's phone alerting them which type of bird is in their garden. For example, an elderly man who loves to bird watch has 2 feeders in his garden at different places around his house. His vision is a bit impaired, so he cannot keep a close watch on the feeders since this costs too much energy. With the automatic birdfeeder, he does not have to keep a close watch since he now knows when and which bird is feeding, giving him a very pleasant user experience.

The automatic birdfeeder can also be used for wildlife conservation in areas where certain species have a difficult time surviving. Many habitats have exotics, species not native to an area, that were taken as pets from different countries and escaped. These exotics pose a big danger to the biodiversity of ecosystems since they do not have natural predators living there. As a result, the numbers of their population increase rapidly, leaving not enough food for native species. With the automatic bird feeder, wildlife reservists can help the native species by giving them food without the exotics benefiting from their advantage. The automatic birdfeeder can also help in helping to understand the severity of the problem by keeping count of the birds that tried to feed. This is also interesting for researchers and ornithologists to preserve ecosystems better since birds play a vital role in almost all ecosystems globally.

For the scope of this course, we decided to focus on the group of people who just like to have a feeder in their garden. To understand their needs better, we have conducted multiple semi-structured interviews with different users. The first questions were open questions regarding peoples bird watching behaviour. The latter questions were more guided by already identified possible features. These features were a way to scare off larger birds, a notification feature, bird identification and overfeeding prevention. The participants gave their thoughts on the proposed features and ranked them from most useful to least useful. The research questions and short summaries of the interviews can be found in Appendix D and Appendix E. From the answers, an affinity diagram was made with five different identified themes.

Requirements

MoSCoW

It is important to note down the exact requirements we have for this project. This gives us a clear goal to work towards and also helps us later to create a test plan. In this section, we note down our requirements for our project. We base these on the affinity attributes mentioned above and rank them according to the MoSCoW Theory. This means that every requirement has a given priority based on how important it is for the final prototype.

These requirements are split up in different categories: physical design, electronics, communication, app design, app user interface, and other app requirements. Most of these requirements are for potential further developments of our project. We thought of them during the initial phase of our project, but we realized that we did not have time to finish most of the requirements in time.

One of the biggest features we decided to scrap is the app. We mentioned it during our interviews and in our initial phase, but we decided that it was outside the scope of our project, because of the time constraints.

MoSCoW

Requirement ID	Description	Priority
PD01	The system shall have a container for bird food	Must have
PD02	The system shall allow the user to hang the system on a hook	Could have
PD03	The system shall allow the user to hang the system using screws	Could have
PD04	The system shall have a landing area for the birds	Should have
PD05	The system shall have a door to the bird food	Must have
PE01	The system shall measure the amount of bird food left in the container	Won’t have
PE02	The system shall record the area in front of the door	Must have
PE03	The system shall detect birds approaching the cage	Must have
PE04	The system shall close the door to the bird food if an undesired bird* has been detected	Must have
PE05	The system shall keep the door to the bird food open if no undesired bird* has been detected	Must have
PE06	The system shall open the door to the bird food if no undesired bird* has been detected and the door is closed	Must have
PE07	The system shall scares away undesired birds	Should have
PE08	The system shall record the number of detected birds	Could have
PE09	The system shall record the species of detected birds	Could have
PE10	The system shall have an internet connection	Must have
PE11	The system shall produce audio to attract desired birds	Could have
PE12	The system shall refill the food if less than 50 grams of food is detected	Won’t have
PE13	An additional power source, like solar panels, shall power the system.	Won’t have
PC01	The system shall send the amount of bird food left in the container to the app	Won’t have
PC02	The system shall send the recorded data to the app	Could have
PC03	The system shall allow the user to connect the system to a registered account	Won’t have
AD01	The app shall display the amount of food left in the container	Won’t have
AD02	The app shall display the number of birds by species detected	Won't have
AD03	The app shall graph the number of birds detected	Won't have
AD04	The app shall graph the amount of food measured	Won’t have
AU01	The app shall allow the user to register a new account	Won’t have
AU02	The app shall allow the user to login to a new account	Won’t have
AU03	The app shall allow the user to connect the account to multiple of our products	Won’t have
AU04	The app shall display general information about bird species	Won't have
AO01	The app shall send a push notification to the user if there is less than 50 grams of food left	Won’t have
AO02	The app shall recommend bird food	Won't have
AO03	The app shall allow the user to export data to a .csv file	Won’t have

Research

Bird luring

Early in this project, we also thought about adding ways to attract birds to our bird feeder. Although we didnt go through with this, as apparent from our requirements, we did research anyway for accomplishing this and found valuable information for future enhancements of our project.

In "Trees, Shrubs and Vines for attracting birds" by DeGraaf, he documents a wide variety of plants and what birds are interested in those plants. He documents whether birds settle near the plant, take shelter near the plant or eat its fruits. We can use this information to add plants to our birdfeeder itself, or guide users in creating a preferable habitat for the birds they would like to atract. Furthermore, McAtee mentions that birds, especially in dry periods, are attracted to water supplies. They like water supplies for both drinking and washing purposes. Again, this information could be used to further enhance our product or to guide users in creating a prefered habitat for birds.

We also hypothesized that mimic bird sounds could attract birds to our product. We could not find any good research that aligned that subject with our project, but we found various discussions across the internet. One of the main concerns about this approach is whether it is an ethical approach or not, as it essentially relies on deceiving birds. Furthermore, it could also have the opposite effect, as it could scare away birds because of territorial behaviour. We thus quickly realized that this is not a feature we want to add to our project.

Design

Classifier (Recognition System)

The classifier is the component in the system responsible for detecting and recognising (classifying) birds. Specifically, the classifier is tasked with classifying whether an image, outputted by the camera, contains a specie of bird which is considered undesirable.

For the classifier, we have used Roboflow. Roboflow is a platform that useful tools for computer vision models. We have also considered using models more directly using the PyTorch library, but this proved more cumbersome in terms of configuration. After some basic testing, Roboflow did not seem to perform significantly worse than PyTorch in classification.

The Raspberry PI is capable of running the Roboflow server through Docker, but our particular model of Raspberry PI (model 4B) seemed to have an older instruction set, on which Roboflow could not be ran. Instead, for our prototype, we have ran the Roboflow server on a connected laptop.

Corner Cases

There are two corner cases regarding birds classification: the system might encounter a bird it cannot classify, or multiple birds might be encountered at one time.

For the first case, if the system encounters a bird that it cannot classify, it will assume that the given bird is not undesirable. This is to prevent any false negatives. False negatives -- closing the door when a potentially desired but unclassified bird is seen -- could have scared off more birds, and, as birds are often reluctant to feed in new spots, we feared we would scare off more birds permanently from our feeder.

For the second case, when the classifier recognizes both a desired and undesired bird, it prioritizes the fact that an undesired bird was encountered, and closes the door. This is because (apparent from our interviews) some birds bully others, and one of the points of our feeder is to minimize this behaviour and attempt to give smaller birds an advantage. Leaving the door opened could have given more opportunity for bullying birds to scare off smaller ones.

Structure

The classifier consists of two parts: a (general) object detector in order to detect where in the image a bird might be, followed by a ViT (Vision Transformer) classifier which classifiers the bird. The object detector outputs a bounding box detailing the position of the recognized birds. The image is cropped to this bounding box, then resized to a default resolution, and finally used as the input to the ViT.

We have used the "YOLOv11" model for the object detector, and trained our own model for the ViT classifier. We have chosen to use an object detector in addition to the simple ViT classifier for two reasons:

Firstly, the camera is angled outwards, viewing a larger area, meaning that the images contain birds at difference distances, and hence different apparent sizes. If the ViT is trained on a dataset where all the birds were photographed from more or less the same distance, then it might give inconsistent output when classifying images with birds from a further distance. Thus, by first cropping the images using the object detector, we could be more certain of consistent outputs.

Secondly, there could be multiple birds in view at one time. In such case, a simple single-output ViT classifier is not sufficient. The object detector can recognize multiple objects in one scene, and each bird recognized by the object detector is classified separately by the ViT, which simplifies the requirements for the ViT, as it can be trained on simple images containing one bird each.

We have chosen for "YOLOv11" simply for the fact that it was the most recent pre-trained object detector model available in Roboflow. Alternatives would have been less recent versions (such as "YOLOv8") or train our own model, but given the scope of the prototype and the fact that the pre-trained model worked well enough, we chose the pre-trained model.

Lastly, we wish to mention the confidence threshold as one of our hyperparameters: we had a confidence threshold of >0.4 for our ViT classifier. This is because after some testing, we had noticed that this suffice in cutting out the additional output. For instance, when classifying a sparrow, the classifier will also always output some small chance of it being crow. These small chances were typically below 0.3, so a cut-off of 0.4 provided a bit of margin on top of that.

Dataset

The dataset was chosen to have four classes, two of which were generally desired birds, namely sparrow and great tit, and two of which were undesired, namely crow and magpie. This limited set of classes was chosen as necessary minimum viable product to display the prototype's ability to classify birds. The birds were chosen from the results of the interviews. Other possible classes would have been "blackbird" or "sea gull".

Each dataset contains eight images, selected from online image searches, which are cropped YOLOv11. Each image is furthermore also copied and rotated +15 or -15 degrees to generate additional training data. The chosen images varied in size and background, and the birds depicted varied in the direction in which they were photographed.

We had also looked at using a pre-trained model for bird classification. Most of the existing models we had looked at, however, seemed to be trained on datasets of North American bird species. Some quick manual tests showed that these pre-trained models were inadequate for classifying common Dutch birds, given they were not present in the respective datasets.

TODO MIHAI: SHOW IMAGES FROM THE DATASET

Door system... todo Luca: check section

One of the two core principles is the door that closes when unwanted birds are at the feeder, so we put a lot of thought into how we wanted to design the door. We primarily evaluated our choices on their mechanical complexity, and how dangerous they would be to birds.

The option we ended up using was a drop-down hatch. Normally, the door would rest in the horizontal position, giving access to the food inside. Then, when the system decides the door needs to be closed, it rotates 90 degrees. Closing into its vertical position, flush with the wall. If the motor turns out to not be precise enough to close flush with the wall every time, we can help with closing correctly by adding a little protrusion in front of the door, which does not allow the door to rotate further. The closing of the door works exactly the same but in reverse. It rotates 90 degrees from its vertical position to its horizontal position allowing access to the food again.

Mechanically, this design is not very complicated. Depending on how you want the aesthetics of the feeder to be, you can either mount the door directly to the motor’s axis, with the motor being mounted on the outside of the feeder, or you can put the motor inside the feeder and use something like gears or chains to rotate the door.

We have chosen to put the motor inside the feeder, but as we do not know the precise layout at the time of making the decisions, the actual implementation is left to design in the modeling stage. We do also think this door is quite bird friendly. The door does not have to close fast, so if there are still birds feeding when the door has to close, they are gently pushed back without risk of getting stuck between the door.

The other option that was considered was a door that rotated in front of the circular hole the food was accessible from. The door would normally be rotated such that it was not blocking the food, either upright or to the side, depending on where the camera would have been placed. Then if the system decided there was an unwanted bird present, it would rotate the door such that it would block the ‘food hole’. Then when the system decides the food might become accessible again, it would reverse this process, rotating the door away from the hole.

Mechanically, this idea is very simple. You can simply mount the door directly to a servo with its rotation axis orthogonal to the front side of the feeder, and then it only has to rotate a specific amount. You could even help the motor by placing stoppers on both sides of the rotation, to prevent the motor from rotating the door too far.

However, we ended up not choosing this design. Even though mechanically it is very simple, we did fear for the safety of the birds.

If the motor is very strong, we fear that birds might get their head stuck and get seriously injured, which is not what we want.

If the motor is just barely strong enough to move the door (Or you have some sort of advanced door-obstruction system), the door will not crush the bird’s head. However, we might fear that the birds are smart enough to figure this out. They might be scared of a moving object closing on them the first few times, but we predict birds (especially bigger birds) will be smart enough to figure out this system is actually harmless, which then renders our entire design useless.

Overall we might have been able to make this door work, but our other idea wasn’t that mechanically complex either, and we preferred how it looked/functioned over this one.

Sequence diagram

Sequence diagram of the Selective Bird feeder

Testing

Test plan

Requirement	Description	Precondition	Action	Expected output	Observed Output	Result
PD01	The system shall have a container for bird food	-	-	Model has a container for birdfood	Model has a container for birdfood	Pass
PE02	The system shall record the area in front of the door	-	Turn on camera and associated code, and view its output.	The camera functions and we have a picture of the area in front of the door
PE03-0	The system shall detect birds approaching the feeder	The camera functions correctly	Run the identifying software on the image the camera provides, and watch its output.	The software correctly identifies there is a bird in the image
PE03-1	The system detects the birds 'mus, koolmees, kraai, ekster'	The system can detect birds approaching the feeder	Run identifying software and seperately show pictures of a mus, koolmees, kraai and ekster	The software correctly identifies each individual bird.
PE04

Discussion

Future work

- Refining prototype

- Different place for components

- Different materials

- Think of other aspects (App, sharing data with institutions, mounting, power, food monitor)

- Run local or centralized

Appendix

Appendix A: Planning

Week number	Task
Week 1	Brainstorming for potential ideas + State of the art research + MoSCoW theory
Week 2	User research + Literature research and patents + design
Week 3	Built feeder + work on user-specified features
Week 4	Implement features into the feeder
Week 5	Finish Prototype
Week 6	Test + improve prototype
Week 7	Finalising product + Finish Wiki + presentation

Appendix B: Sources

https://books.google.nl/books?hl=en&lr=&id=H3Cx1hB-TeQC&oi=fnd&pg=PR9&dq=attracting+birds&ots=yf6QfmbysH&sig=-EyO28YygyhS2iJluR8gjbLpYnM#v=onepage&q=attracting%20birds&f=false

https://books.google.nl/books?hl=en&lr=&id=qLd1mSAMSz0C&oi=fnd&pg=PP7&dq=attracting+birds&ots=aX5OLryMyA&sig=d5jGRLf54bOp6OquQYQmWINXZZk#v=onepage&q=attracting%20birds&f=false

Week 1:

Lists of interests for each group member.

We will use this to decide our final pick(s).

Jort	Leander	Luca	Pijke	Thijs
- Drugs test robot for in a bar to prevent people from getting drugged. - Safety walk system in which you can set you designation and it looks for safest road with most lights and busyness.Warning when straying from path - Social interaction robot which moves while video chatting	- Robotic "guidedog" for visually impaired (something that detects obstacles and communicates avoidance directions) - AI bird feeder (identifies, dispenses accordingly) - AI window cleaner (cleans windows that are hard to reach for example, finds them and cleans them)	- app to chessboard - Pill dispenser - smart energy meter and app	- Software-hardware connection - Making a robot move; pathfinding - Visual recognition; Sorting pills?	-Embedded systems -Solving puzzles -Entertainment or usefulness