Firefly Eindhoven - Remaining Sensors

IMU

Inertial Measurement Unit or IMU is a sensor board on the drone which is use to measure the orientation of the drone. The IMU consists of three sensors:

• Accelerometer
• Gyroscope
• Magnetometer

The accelerometer measures the acceleration of the object, using which the orientation of the drone can be estimated from the direction of the gravitation vector. The gyroscope measures the angular velocity of the object from which the angular displacements can be calculated. Even though these are some of the very standard sensors for UAV's, the accelerometer is very sensitive to any external forces: presence of external forces influences the measurement accuracy of the sensor. As for the gyroscope, the measurements of the gyroscope are usually quite accurate after calibration but however ,as time progresses, the angular displacement estimation becomes very inaccurate as it accumulates the integrated error and drift in velocity measurement.

This however, can be fixed by use of sensor fusion algorithms.

Optical flow

Optical flow refers to estimation of apparent velocities of certain objects in an image. This is done by measuring the optical flow of each frame using which velocities of objects can be estimated. It is 2D vector field where each vector is a displacement vector showing the movement of points from first frame to second. By estimating the flow of points in a frame, the velocity of the moving camera can be calculated. The estimation of velocity allows the use of advanced control schemes for the drone such as LQR because simple controllers such as PID just differentiate the position to damp the system, but this introduces amplification of noise within the control loop. A technique like LQR would directly make use of the velocities and does not have this problem.

Working Principle

If [math]\displaystyle{ I(x,y,t) }[/math] is the intensity of a pixel in an image then after some time [math]\displaystyle{ dt }[/math], as the pixel moves some distance [math]\displaystyle{ dx }[/math] and [math]\displaystyle{ dy }[/math] then as the pixel intensity is consistent, it can be said that;

[math]\displaystyle{ I(x,y,t) = I(x+dx, y+dy, t+dt) }[/math]

Using taylor series, it is possible to write

[math]\displaystyle{ \frac{\partial I}{\partial x} \frac{\partial x}{\partial t} + \frac{\partial I}{\partial y} \frac{\partial y}{\partial t} + \frac{\partial I}{\partial t} = 0 }[/math]

In the above equation, the space differentials of intensity are refereed to as image gradients and the above equation is termed as Optical flow Equation. The time differentials of the pixel position [math]\displaystyle{ x, y }[/math] are the unknowns which determine the optical flow.

There are multiple algorithms in order to solve this problem, the most popular ones being:

• Lucas Kanade Method
• Phase Correlation

Open MV Camera

In order to perform optical flow estimation, the team decided to use the Open MV Camera. The Open MV Camera can be programmed via the Open MV IDE. The language used in the IDE is Python with some additional libraries that are meant only for Open MV. In addition, Open MV provides multiple examples which can directly be used to solve the problem.

The optical flow in the open mv can be achieved by the 'displacement' function in the open mv ide. This function makes use of the phase correlation method to estimate the optical flow. However due to limited computational capacity of the on-chip processor the algorithm first reduced the quality of the images captured and then performs the velocity estimation.

In order to estimate the velocities of the drone, it was decided that a few LEDs would be placed on the ground and by meauring the optical flow of the LEDs, the velocties of the drone could be measured. However, as the camera quality of the Open MV was very limited and this was even more deteriorated by the algorithm, we decided to modify the existing code.

In order to reduce the computational power consumption, we used a blob detection that would detect the LED's and perform the phase correlation on that, this would limit the number of pixels over which phase correlation was performed.

However, after multiple tests and trials it was determined that the hardware of the Open MV camera would make optical flow possible in very specific and static conditions and our conditions for the show were very dynamic, therefore the team decided to drop the idea for the moment.