PRE2018 3 Group11: Difference between revisions

From Control Systems Technology Group
Jump to navigation Jump to search
No edit summary
Line 204: Line 204:
# A Graphical User Interface displayed through a web page that provides enter or change the destination and current location, add favorite locations for quick access. Allows user to change the altitude of the drone, stop and start the drone. Search bar: search for place or address, icons buttons that say food and drink, shopping, fun, and travel. Pressing food and drink shows a list of restaurants and bars and cafes with yelp reviews. Pressing travel shows gas stations, landmarks, airports, bus stations, train stations, hotels. Pressing fun shows cinemas and other entertaining places. Favorites show frequently used addresses.
# A Graphical User Interface displayed through a web page that provides enter or change the destination and current location, add favorite locations for quick access. Allows user to change the altitude of the drone, stop and start the drone. Search bar: search for place or address, icons buttons that say food and drink, shopping, fun, and travel. Pressing food and drink shows a list of restaurants and bars and cafes with yelp reviews. Pressing travel shows gas stations, landmarks, airports, bus stations, train stations, hotels. Pressing fun shows cinemas and other entertaining places. Favorites show frequently used addresses.
# A Motion User Interface. Can interpret motion gestures.
# A Motion User Interface. Can interpret motion gestures.
== Distance Estimation ==


== Obstacle avoidance ==
== Obstacle avoidance ==
Line 268: Line 271:
=== Line of Sight ===
=== Line of Sight ===
Whereas the experimental evaluation of the trackers was done to find one that can continue to track even with bad visibility or obstacles, there is also a need of some piece of code that can say when the drone is no longer in line of sight of the user. Instead of coming up with a clever computer vision algorithm to do so, we can simply use one of the trackers that performs bad on tracking the user when there are obstacles. Whenever said tracker (which is going to be KCF) says that it can no longer track the object, which is the user, then we can state that the drone is no longer in line of sight. The reason why KCF was chosen over BOOSTING, is because it is the better tracker in general.  
Whereas the experimental evaluation of the trackers was done to find one that can continue to track even with bad visibility or obstacles, there is also a need of some piece of code that can say when the drone is no longer in line of sight of the user. Instead of coming up with a clever computer vision algorithm to do so, we can simply use one of the trackers that performs bad on tracking the user when there are obstacles. Whenever said tracker (which is going to be KCF) says that it can no longer track the object, which is the user, then we can state that the drone is no longer in line of sight. The reason why KCF was chosen over BOOSTING, is because it is the better tracker in general.  
== User Height Computation ==
We describe why an implementation would need some information that was not present in dorm-conditions and how to possibly overcome that.
Height estimation and (relative) distance estimation may seem like two separate things, but one is needed for the other as we will show in this part of the wiki. First we will talk about user height estimation.
In order for the (software of the) drone to estimate the height of the user, which is needed in order to find the correct flying place, we need rather many pieces of information. Firstly, there is the type of lens used in the camera on the drone. We are going to assume it is a rectilinear lens, since the majority of lenses made are rectilinear [https://www.borrowlenses.com/blog/rectilinear-fisheye-wide-angle-lens/]. Note that the following calculations are also possible for other types of lenses, though may vary.
<!-- TODO image -->
As shown on the diagram, there are various parameters that play a role in height computation:
# <math>h_{sensor}</math> : The height of the sensor.
The height of the sensor is generally unknown. It can, however, be computed rather easily in terms of the sensor’s pixel-size (which should be on the spec sheet of a well documented camera) and the resolution of the image (which can be very easily found using an OpenCV function) [http://www.ni.com/product-documentation/54616/en/].
# <math>h_{obj}</math> : The height of the object in pixels as shown in the image (video).
While this piece of information (or parameter) is unknown, we can simply set it equal to the bounding box that is returned by an object tracker.
# <math>f</math> : The distance from the lens to the sensor (focal length).
The focal length should be visible on the spec sheet of a well documented lense-camera system. If not, it can be computed, but it requires item 6, the distance from the lens to the object in order to be computed [http://www.ni.com/product-documentation/54616/en/].
# <math>\alpha</math> : The angle of the lense.
While this is not necessarily a parameter, it is helpful to be mentioned. Alpha is the (vertical) angle that the lens makes. This will not be the same for every type of lense, hence our assumption that the lense is rectilinear. Furthermore, it is used to compute the FOV (Field of View) as will be shown later.
# <math>\theta</math> : The angle that the object takes up.
This will be factored out as shown later in the computation.
# <math>D</math> : The distance from the lens to the object (user)
The distance from the lens to the object is unknown. There are broadly speaking two possible ways to get this distance. Firstly, there is computer vision. Using an object marker, one can compute the distance to said marker given two major unknown parameters:
* The size of the object marker (it needs to be a known object).
* The distance that the camera was when it registered the object marker.
Clearly, this is not feasible, leaving only option two.
[https://www.pyimagesearch.com/2015/01/19/find-distance-camera-objectmarker-using-python-opencv/]
This means that one should use sonar to estimate the depth [https://brage.bibsys.no/xmlui/bitstream/handle/11250/2394445/11809_FULLTEXT.pdf?sequence=1]. The linked paper explains in great detail how it works, but mentions that it is a slow algorithm. Luckily, it also mentions that is has been implemented in Matlab, and since matlab is very slow when compared to C++ [https://pdfs.semanticscholar.org/ed2b/5f9a2ca37e55052eafbf5abc166245cf7995.pdf] it is fair to assume that, when implementing in C++, the used algorithm as proposed in the paper is fast enough to compute the distance every 5 seconds.
# <math>H_{obj}</math> : The (real-world) height of the object (user).
This is what you want to know.
# <math>H_{scene}</math> : The (real-world) height of the scene (image).
This is irrelevant to the computation, but mentioned for completeness sake.
As for the computation, one starts with rewriting the angles in radians, leading to following equations.:
* <math>D \cdot \theta = H_{obj} (1) </math>
* <math>f \cdot \theta = h_{obj} (2) </math>
Using (1) and (2) to factor out <math>\theta</math> gives:
* <math> \frac{D}{f} \cdot h_{obj} = H_{obj} (3)</math>
Note that in (3) the unit of <math>h_{obj}</math> is in pixels, whereas the unit of <math>H_{obj}</math> is assumed to be in the metric system. Converting pixels to millimeters is done as follows:
* <math> h_{obj}(mm) = \frac{h_{obj}(px)}{h_{sensor}(px)} \cdot h_{sensor}(mm) (4)</math>
Rewriting (3) and (4) gives the following (rewriting focal length to mm):
* <math>D \cdot \frac{h_{obj}(px)}{h_{sensor}(px)} \cdot \frac{h_{sensor}(mm)}{f(mm)} = H_{obj} (5)</math>
Using (5), depending on which information you have available, you can compute <math>H_{obj}</math> directly. Otherwise, some more work is needed (especially if <math>f</math> is unknown):
* <math>\frac{h_{sensor}(mm)}{f(mm)} = \alpha(radians) (6)</math>
This is, as can be seen in (6) where <math>\alpha</math> comes in. Note again that this means the vertical angle that the lense makes, in other words, the vertical Field of View ( <math>\alpha = V_{FOV} </math>). Note that you may have to convert between radians and degrees as needed. Finally, rewriting (5) using (6) gives.
* <math>H_{obj} = D \cdot \frac{ h_{obj}(px)}{h_{sensor}(px)} \cdot V_{FOV}(radians) (7)</math>
Using (possible parts of) these 7 equations, one can compute the height of an object, thus of a user.


= Simulation =
= Simulation =


= Conclusion =
= Conclusion =

Revision as of 17:58, 27 February 2019

<link rel=http://cstwiki.wtb.tue.nl/index.php?title=PRE2018_3_Group11&action=edit"stylesheet" type="text/css" href="theme.css"> <link href='https://fonts.googleapis.com/css?family=Roboto' rel='stylesheet'>

  1. Home
  2. Organization
  3. Brainstorm


Organization

The group composition, deliverables, milestones, planning and task division can be found on the organization page.

Brainstorm

To explore possible subjects for this project, a brainstorm session was held. Out of the various ideas, the Follow-Me Drone was chosen to be our subject of focus.

Problem Statement

The ability to navigate around one’s environment is a difficult mental task whose complexity is often dismissed by most of us. By ‘’difficult’’, we are not referring to tasks such as pointing north while blindfolded, but rather much simpler ones such as: following a route drawn on a map, recalling the path to one’s favourite cafe. Such tasks require complex brain activity, which causes the broad skill of ‘navigation’ to be present in humans in diverse skill levels [citation needed]. People whose navigation skills fall on the lower end of the scale do seem to be slightly troubled in their everyday life; it might take them substantially longer than others to be able to find their way to work through memory alone; or they might get lost in new environments every now and then. However, it is important to note, that this is a dynamic grey-scale; there is no line between ‘good’ and ‘bad’ navigation.

Navigation skills can be traced back to certain distinct cognitive processes (to a limited extent, nevertheless, as developments in neurosciences are still progressing [1]), which means that a lesion in specific brain regions can cause a human’s navigational abilities to decrease dramatically. This was found in certain case studies with patients that were subject to brain damage, and labeled “Topographical Disorientation” (TD), which can affect a broad range of navigational tasks [2]. Within the last decade, cases started to show up, in which people showed similar symptoms to patients with TD. However, they did not suffer any brain damage and were normally functioning in all other aspects. This disorder was termed “Developmental Topographical Disorientation” (DTD) [3], which, as was recently discovered, might affect 1-2% of the world's population (it is important to note that this is an estimate based on studies and surveys) [4].

Many of the affected people cannot rely on map-based navigation and sometimes even get lost on everyday routes. To make their everyday life easier, one solution might be to use a drone. The idea of the playfully-named “follow-me drone” is simple: By flying in front of its user, accurately guiding them to a desired location, the drone takes over the majority of a human’s cognitive wayfinding tasks. This makes it easy and safe for people to find their destination with minimized distraction and cognitive workload of navigational tasks.

The first design of the “follow-me drone” will be mainly aimed at severe cases of TD, with the goal of giving those people a little bit of their independence back. This means that the design incorporates features that might make the drone more valuable to its user with repeated use and is not only thought-of as a product for rare or even one-time use as part of a service.


User

Society

The societal aspect of the follow-me drone regards the moral standpoint of society towards cognitively-impaired individuals. Since impairments should not create insuperable barriers in the daily life of affected people, it can be seen as a service to society as a whole to create solutions which make it easier to live—even if that solution only helps a small fraction of the world's population.

Enterprise

Since this solution is unique in its concept and specifically aimed at a certain user group, producing the drone is a worthy investment opportunity. However, the focus should always stay on providing the best possible service to the end-users. User-centered design is a key to this development.

Approach

In order to get to a feasible design solution, we will do research and work out the following topics:

  • User requirements
    • Which daily-life tasks are affected by Topographical Disorientation and should be addressed by our design?
    • What limitations does the design have to take into account to meet the requirements of the specified user base?
  • Human technology interaction
    • What design factors influence users’ comfort with these drones?
    • Which features does the technology need to incorporate to ensure intuitive and natural guiding experiences?
      • How to maximise salience of the drone in traffic?
      • What velocities, distances and trajectories of the drone will enhance the safety and satisfaction of users while being guided?
    • Which kind of interfaces are available in which situations for the user to interact with the drone?
  • User tracking
    • How exact should the location of a user be tracked to be able to fulfill all other requirements of the design?
    • How will the tracking be implemented?
  • Positioning
    • How to practically implement findings about optimal positions and trajectories?
  • Obstacle avoidance
    • What kind of obstacle avoidance approaches for the drone seem feasible, given the limited time and resources of the project?
    • How to implement a solution for obstacle avoidance?
  • Special circumstances
    • What limitations does the drone have in regards to weather (rain, wind)?
    • How well can the drone perform at night (limited visibility)?
  • Physical design
    • What size and weight limitations does the design have to adhere to?
    • What sensors are needed?
    • Which actuators are needed?
  • Simulation

Alternative Solutions

Solution

Here we discuss our solution. If it exists of multiple types of sub-problems that we defined in the problem statement section, then use separate sections (placeholders for now).

Requirements

Aside from getting lost in extremely familiar surroundings, it appears that individuals affected by DTD do not consistently differ from the healthy population in their general cognitive capacity. Since we can't reach the targeted group, it is difficult to define a set of special requirements.

  1. The Follow-Me Drone has a navigation system that computes the optimal route from any starting point to the destination.
  2. The drone must guide the user to his destination by taking the decided route hovering in direct line of sight of the user.
  3. The drone must guide the user without the need to display a map.
  4. The drone must provide an interface to the user that allows the user to specify their destination without the need to interact with a map.
  5. The drone announces when the destination is reached.
  6. The drone must fly in an urban environment.
  7. The drone must keep a minimum distance of two? meters to the user.
  8. The drone must not go out of line of sight of the user.
  9. The drone must know how to take turns and maneuver around obstacles and avoid hitting them.
  10. The drone should be operable for at least an hour between recharging periods back at home.
  11. The drone should be able to fly.
  12. (Optional, if the drone should guide a biker) The drone should be able to maintain speeds of 10m/s.
  13. The drone should not weigh more than 4 kilograms.



Human factors

Drones are finding more and more applications around our globe and are becoming increasingly available and used on the mass market. In various applications of drones, an interaction with humans who might or might not be involved in the task that a drone carries out, is inevitable and sometimes even the core of the application. Even though drones are still widely perceived as dangerous or annoying, there is a common belief that they will get more socially accepted over time [5]. However, since the technology and therefore the research on human-drone interaction is still very new, our technology should incorporate as many human factors as deemed necessary, without assuming general social acceptance of drones.

The next sections will focus on different human factors that will influence our drone’s design with different goals. This will include the drone’s physical appearance and its users’ comfort with it, safety with respect to its physical design, as well as functionality-affecting factors which might contribute to a human's ability to follow the drone. Additionally, factors contributing to a satisfying overall experience will be considered.

Physical appearance

Here, the close physical appearance of the drone and its users affect for it will be discussed. Size: Majority in studies for medium size drone (around 40cm diameter) [6] Shape: Round features


Safety

The safety section will cover topics like the minimum distance it should keep to its user and eventually physical design guidelines to be able for users to handle the drone safely.

Salience in traffic

As our standard use case described, the drone is aimed mostly at application in traffic. It should be able to safely navigate its user to his destination, no matter how many other vehicles, pedestrians or further distractions are around. A user should never get the feeling to be lost while being guided by our drone.

To reach this requirement, the most direct solution is constant visibility of the drone. Since human attention is easily distracted, also conspicuity, which describes the property of getting detected or noticed, is an important factor for users to find the drone quickly and conveniently when lost out of sight for a brief moment. However, the conspicuity of objects is perceived similarly by (almost) every human brain, which introduces the problem of unwillingly distracting other traffic participants with an overly conspicuous drone. We will focus on two factors of salience and conspicuity:

  • Color

The color characteristics of the drone can be a leading factor in increasing the overall visibility and conspicuity of the drone. The more salient the color scheme of the drone proves to be, the easier it will be for its user to detect it. Since the coloring of the drone alone would not emit any light itself, we assume that this design decision does not highly influence the distraction factor for other traffic participants.

Choosing a color does seem like a difficult step in a traffic environment, where many colors have meanings that many humans take for granted. We would not want the drone to be confused with any traffic lights or street signs, since that would lead to serious hazards. But we would still need to choose a color that is perceived as salient as possible. Stephen S. Solomon conducted a study about the colors of emergency vehicles [7]. And the results of the study were based upon what was found out about the human visual system: It is most sensitive to a specific band of colors, which involves ‘lime-yellow’. Therefore, as the study showed, lime-yellow emergency vehicles were involved in less traffic accidents, which does indeed let us draw conclusions about the conspicuity of the color. This leads us to our consideration for lime-yellow as base color for the drone. Furthermore, color contrasts are going to be discussed here.

  • Luminosity

The luminosity section will treat the application of reflectors and lights on the drone, maximising visibility but minimising unwilling conspicuity for other road users.

Positioning in the visual field

The positioning of the drone will be as valuable for a user as it will be a challenge to correctly derive. The drone should not deviate too much from its usual height and position within its users attentional field, however, it might have to avoid obstacles, wait and make turns without confusing the user. Findings so far:

  • Smooth movements (planned trajectories)
  • Horizontal deviation better than vertical deviation [8]


User satisfaction and experience design

A user should feel comfortable using this technology. This section will treat how this could be achieved and how a user’s satisfaction with the experience could be increased.

Legal Issues

In this project, a flying drone is designed to move autonomously through urban environments to guide a person to their destination. It is important however to first look at relevant legal issues. In the Netherlands, laws regarding drones are not formulated in a clear way. There are two sets of laws: recreational-use laws and business-use laws. It is not explicitly clear which set of laws applies to this project if any at all. The drone is not entirely intended for recreational use because of its clear necessity to the targeted user group. On the other hand, it does not fall under business use either it is not “used by a company to make money”---this does not include selling it to users. In that case, it is in the users’ ownership and is no longer considered used by the company. If the drone is rented, however, it still might fall under business use. In conclusion, neither set of laws applies to our project. Since drones are not typically employed in society, this is not unexpected and we might expect new laws to adapt to cases such as ours.

According to law, drone users are not allowed to pilot drones above residential areas regardless of their use. Our drone, however, flies through residential areas for its intended purpose, not above them.

Legislation is subject to change that can be in accordance with our project as well as against it. That being said, we can safely design our drone as intended without worrying about legislation. Our design does not explicitly violate any laws.

Technical considerations

Battery life

One thing to take into consideration while developing a drone is the operating duration. Most commercially available drones can not fly longer than 30 minutes [9],[10]. This is not a problem for a walk to the supermarket around the block, but to commute for example to work longer battery life would be preferred. Furthermore wind and rain have a negative influence on the flight time of the drone but the drone should still be usable during these situations so it is important to improve battery life.

In order to keep the battery life as long as possible, a few measures could be taken. First of all a larger battery could be used. This however makes the drone heavier and thus also would require more power. Practical energy density of a lithium-polymer battery is 150Wh/kg [11]. However if the battery is discharged completely it will degrade, decreasing its lifespan. Therefore the battery should not be drained for more than 80% meaning that the usable energy density is about 120Wh/kg.

Another way to improve flight time is to keep spare battery packs. It could be an option in this project to have the user bring a second battery pack with them for the trip back. For this purpose the drone should go back to the user when the battery is nearly low and land. The user should then pick the drone up and swap the batteries. This would require the batteries to be easily (dis)connectable from the drone. A possible downside to this option is that the drone would temporarily shut down and might need some time to start back up after the new battery is inserted. This could be circumvented by using a small extra battery in the drone that is only used for the controller hardware.

For this project first it is researched whether a continuous flight of around 60 minutes is feasible. If this is not possible it is chosen to use the battery swapping alternative. Unfortunately no conclusive scientific research is found about the energy consumption of drones so instead existing commercially available drones are investigated. In particular the Blade Chroma Quadcopter Drone [12] is researched. It weighs 1.3kg and has a flight time of 30 minutes. It uses 4 brushless motors and has a 11.1V 6300mAh LiPo battery. The amount of energy can be calculated with [math]\displaystyle{ E[Wh]=U[V]\cdotC[Ah] }[/math] which gives about 70Wh. The battery weighs about 480g [13] so this seems about right with previously found energy density. The drone has a GPS system, a 4K 60FPS camera with gimball and a video link system. The high-end camera will be swapped for a lighter simpler camera that uses less energy since the user tracking and obstacle avoidance work fine with that. The video link system will also not be used. The drone will only have to communicate simple commands to the user interface. This does spare some battery usage but on the other hand a computer board will be needed that runs the user tracking, obstacle avoidance and path finding. For now it is assumed that the energy usage remains the same. Unfortunately the flight time of 30 minutes is calculated for ideal circumstances and without camera. The camera would decrease the flight time by about 7 minutes . For the influence of weather more research is needed but for now it is assumed that the practical flight time is 15-20 minutes.

If a flight time of 60 minutes is required that would mean that a battery at least 3 times as strong is needed. However since these battery packs weigh 480g, that would increase the drone weight almost by a factor of 2. This would in itself require so much more power that even more battery and motor power is required. Therefore it can be reasonably assumed that this is not a desirable option. Instead the user should bring 2 or 3 extra battery packs to swap these during longer trips.

It is however still possible to design a drone that has sufficient flight time on a single battery. Impossible aerospace claims to have designed a drone that has a flight time of 78minutes whit a payload [14]. To make something similar a large battery pack and stronger motors would be needed.

Propellers and motors

To have a starting point which propellers and motors to use, the weight of the drone is assumed. Two cases are investigated. One is where the drone has flight time of 15-20 minutes and its battery needs to be swapped during the flight. In this case the drone is assumed to have a weight of approximately 1.5kg which is slightly heavier than standard for commercially available long flight time drones. The other case is a drone that is designed to fly uninterupted for one hour. The weight for this drone is assumed to be 4kg, the maximum for recreational drones in the Netherlands.

Multi charge drone with interchangeable batteries

With a weight of 1.5kg the thrust of the drone would at least have to be 15N. However, with this thrust the drone would only be able to hover and not actually fly. Several fora and articles on the internet state that for regular flying (no racing and acrobatics) at least a thrust twice as much as the weight is required (e.g. [15] and [16]) so the thrust of the drone needs to be 30N or higher. If four motors and propellers are used, this comes down to 7.5N per motor-propeller pair.

APC propellers has a database with the performance of all of their propellers [17] which will be used to get an idea of what kind propeller will be used. There are way to many propellers to compare individually so as starting point a propeller diameter of around the 10" is used (the propeller diameter for the perviously discussed Blade Chroma Quadcopter) and the propeller with the lowest required torque at a static thrust (thrust when the drone does not move) of 7.5N or 1.68lbf is searched. This way the 11x4 propeller is found. It has the required thrust at around 6100RPM and requires a torque of approximately 1.03in-lbf or 11.6mNm.

With the required torque and rotational velocity known a suiting motor can be chosen.

User Interface Design

The user interacts with the drone via multiple interfaces. Here we list the specific design elements and dynamics of each of these interfaces.

  1. A Natural Language User Interface (NLUI) to answer questions, make recommendations on desired destinations, take permission to change route mid flight, and similar speech-based interactions.
  2. A Graphical User Interface displayed through a web page that provides enter or change the destination and current location, add favorite locations for quick access. Allows user to change the altitude of the drone, stop and start the drone. Search bar: search for place or address, icons buttons that say food and drink, shopping, fun, and travel. Pressing food and drink shows a list of restaurants and bars and cafes with yelp reviews. Pressing travel shows gas stations, landmarks, airports, bus stations, train stations, hotels. Pressing fun shows cinemas and other entertaining places. Favorites show frequently used addresses.
  3. A Motion User Interface. Can interpret motion gestures.

Distance Estimation

Obstacle avoidance

In order for a drone to navigate effectively through an urban environment, it must circumvent various obstacles that such an environment entails. To name some examples: traffic lights, trees and tall vehicles may form an obstruction for the drone to follow its route. To solve this problem, we will apply a computationally simple method utilizing a potential field [18].

The goal of this method is to continuously adjust the drone’s flightpath as it moves towards a target, so that it never runs in to any obstacles. For this subproblem, we assume the position and velocity of the drone, target and obstacles are known.

The potential field is constructed using attractive and repulsive potential equations, which pull the drone towards the target and push it away from obstacles. The attractive and repulsive potential fields can be summed together, to produce a field guides the drone past multiple obstacles and towards the target.

We consider potential forces to work in the x, y and z dimensions. To determine the drone’s velocity based on the attractive and repulsive potentials, we use the following functions:

[math]\displaystyle{ p_d^{att}(q_d, p_d) = \lambda_1 d(q_d, q_t) + p_t + \lambda_2 v(p_d, p_t)\\ p_d^{rep}(q_d, p_d) = -\eta_1 \dfrac{1}{d^3(q_o, q_d)} - \eta_2 v(p_o, p_d) }[/math]

where [math]\displaystyle{ \lambda_1, \lambda_2, \eta_1, \eta_2 }[/math] are positive scale factors, [math]\displaystyle{ d(q_d, q_t) }[/math] is the distance between the drone and the target and [math]\displaystyle{ v(p_d, p_t) }[/math] is the relative velocity of the drone and the target. Similarly, distance and velocity of an obstacle [math]\displaystyle{ o }[/math] are used.

There may be multiple obstacles and each one has its own potential field. To determine the velocity of the drone, we sum all attractive and repulsive velocities together:

[math]\displaystyle{ p_d(q_d, p_d) = p_d^{att} + \sum_o p_d^{rep} }[/math]

User Tracking

It should be clear that tracking the user is a large part of the software for the Follow-Me drone.We present Python code that can be used with a webcam as well as normal video files, which can use various tracking algorithms.

Tracker Comparisons

Here we give a (short) overview of the different trackers used and their performance with respect to obstacles and visibility. The videos that were used were taken from [19]. The overview will be on a video-basis, that is, we discuss different trackers in terms of each video used.


Initial comparison --- american_pharoah.mp4

This video is from a horse race. For each tracker we selected the horse in the first position at the start of the video as object to be tracked. During the video, the horse racers take a turn, making the perspective change. There are also small obstacles.

  • BOOSTING tracker: The moment that there is a white pole (obstacle), it completely loses track of its target. This tracker cannot be used to keep track of the target when there are obstacles. It can, however, be used in parallel, sending a signal that the drone is no longer in line of sight with the user.
  • MIL tracker: Manages to keep track of the box despite the obstacles. It runs at around 4 to 5 FPS in general, so it might be too slow to apply. It is however fairly accurate.
  • KCF tracker: Similarly to the BOOSTING tracker, the KCF tracker cannot handle obstacles. It is however very fast compared to both BOOSTING and MIL. Hence, for sending a signal that the drone is no longer in line of sight can be done with KCF instead of BOOSTING.
  • TLD tracker: This tracker, while initially thought to be well-performing, runs extremely slow at 2 to 4 FPS. It is inferior to the MIL tracker in every possible way (for this video). Furthermore, there are many false positives, making this a bad choice.
  • MEDIANFLOW tracker: This tracker behaved nearly identical to the KCF tracker; The moment when there is a slight obstacle, it fails to detect and continues to fail. It could be used instead of KCF to signal when a user is no longer in line of sight.
  • MOSSE tracker: Even though it is not formal, sharing our initial notes (slightly edited to avoid cursing) on the MOSSE tracker best describes it: <quote>Whoa are you serious?! Runs at a minimum of 500 FPS (e.g. real-time) and can track the selection perfectly?? This is beautiful. Even the camera-shake is no problem. Changing the perspective is no problem either. This seems like the one to use for now</quote>. Clearly, the MOSSE tracker performs extremely well.
  • CSRT tracker: The moment that there is an obstacle, it follows the obstacle. During the rest of the video, it tries to recover but fails spectacularly, tracking random parts on top of the video.

At this point it has been decided that only the MIL, TLD and MOSSE trackers can be used for actual tracking purposes. For the next video, only these trackers were compared as a final comparison, even though it should be clear that MOSSE is the best one for our use case.

Final comparison --- dashcam_boston.mp4

This video is taken from a dashcam in a car, in snowy conditions. The car starts off behind a traffic light, then accelerates and takes a turn. The object to track has been selected to be the car in front.

  • MIL tracker: Starts off tracking correctly. Runs slow (4-5 FPS). Does not care about the fact that there is snow (visual impairment) or the fact that the camera angle is constantly changing in the turn. Can perfectly deal with the minimal obstacles occuring in the video.
  • TLD tracker: It runs both slower and inferior to MIL. There are many false positives. Even a few snowflakes were sometimes selected as object-to-track.
  • MOSSE tracker: Unsurprisingly, the MOSSE tracker runs at full speed and can track without problems. It does, however, make the bounding box slightly larger than initially indicated

Tracker conclusion

It's pretty clear that the MOSSE tracker is by far the best one. For the remaining videos, we are simply going to run MOSSE and write down any things that it does weird.

  • drone.mp4: This is footage from a drone following a car. MOSSE has no problems whatsoever tracking the car.
  • nascar_01.mp4: Video of a car race at high speed. The bounding box does not stay correct, but it flawlessly tracks the selected car (both one that stays in view as one that leaves the view).
  • nascar_02.mp4: Identical video to nascar_01. No comments needed.
  • race.mp4: Video of people running on a track. There are no real tracking problems.

As can be read, the MOSSE tracker performs well. Using a webcam, selecting various objects to be tracker, the MOSSE tracker can track anything without too many problems. As far as tracking where the user is, MOSSE is the way to go. There is, however, one caveat. The MOSSE tracker (and any other tracker) fails to track one of two very similar objects.

Line of Sight

Whereas the experimental evaluation of the trackers was done to find one that can continue to track even with bad visibility or obstacles, there is also a need of some piece of code that can say when the drone is no longer in line of sight of the user. Instead of coming up with a clever computer vision algorithm to do so, we can simply use one of the trackers that performs bad on tracking the user when there are obstacles. Whenever said tracker (which is going to be KCF) says that it can no longer track the object, which is the user, then we can state that the drone is no longer in line of sight. The reason why KCF was chosen over BOOSTING, is because it is the better tracker in general.

Simulation

Conclusion

Retrieved from "https://dsdwiki.wtb.tue.nl/index.php?title=PRE2018_3_Group11&oldid=67461"