Implementation MSD19
Getting started with Crazyflie 2.X
Need introduction here.
Manual Flight
This section explains in detail on how to setup a Crazyflie 2.X drone starting from hardware assembly to first manual flight. We used Windows to continue with initial setup of the software part for manual flight. However, Linux (Ubuntu 16.04) was preferred for the autonomous flight. Following additional hardware is required to setup first manual flight.
- Bitcraze Crazyradio PA USB dongle
- A remote control (PS4 Controller or Any USB Gaming Controller)
This link was used to get started with assembly and setting up the initial flight requirements
NOTE: It must be noted that the Crazyflie is running on its latest firmware. The steps to flash the Crazyflie with latest firmware are discussed here.
The Crazyflie client is used for controlling the Crazyflie, flashing firmware, setting parameters and logging data. The main UI is built up of several tabs, where each tab is used for a specific functionality. We used this link to get started with first manual flight of the drone and to develop an understanding of Crazyflie Client. Assisted flight mode is recommended as an initial manual testing mode which is explained here. But before this assisted flight mode is executed, loco-positioning system needs to be setup as explained in next section.
Autonomous Flight
To get started with autonomous flight, the pre-requisites are as follows:
- Loco Positioning System: Please refer to “Loco Positioning System Section” for more details. The link explains the whole setup of loco-positioning system.
- Linux (Ubuntu 16.04)
- Python Scripts: We used autnomousSequence.py file from crazyflie python library. We found this script to be the starting point of autonomous flight.
This link was also used develop a comprehensive understanding of autonomous flight. After setting up the LPS, modifications in the python script of autonomousSequence.py file were made to adjust it according to desired deliverables.
Modifications
The following modifications were made in the autonomousSequence.py file.
- Data Logging of Crazyflie Position from Path Planning and Simulator to CSVs
- Path Planning
These modifications can be observed by comparing the original autonomousSequence.py file with our file here. The difference is easily visible and understandable. This is the git-repository of our work. The modified autnomousSequence.py file can be accessed in Motion Control folder.
Troubleshooting
The drone positioning parameters i.e. roll, pitch and yaw are not stable. The solution is to trim the parameters which can be seen in Firmware Configuration headline in the following link. For autonomous flights, issues like updating the anchor nodes and assigning them identification numbers may appear in Windows, but they can be solved by updating the nodes in Linux.
Localization
Need introduction here.
Local Positioning System (Loco Deck)
The core of localization tool used for the drone is based on a purpose-built Loco Positioning system (LPS) that provides a complete hardware and software solution by Bitcraze. This system comes with a documentation regarding installation, configuration, and technical information in the online directory. Before starting, the following preparations are made:
- 1 Loco positioning deck installed on Crazyflie
- 8 Loco positioning nodes positioned within the room
- Node fixed on stands 3D printed from makefile
- USB power source for anchors
- Ranging mode chosen as TWR
Overall hovering ability observed during hardware testing matches with what has been stated in company measurements as +/- 5 cm range.
Laser Range and Optical Sensor (Flow Deck 2.0)
An additional positioning component is a combination of VL53L0x Time of Flight sensor that measures ground distance and a PMW3901 optical flow sensor measures that ground planar movement. The expansion deck is plug and play. However, technical information is available.
Loco Deck + Flow Deck
Integration of sensor information occurs automatically as drone firmware can identify which types of sensors are available. Subsequently, the governing controller handles the data accordingly. To accept more sensor input, the Crazyflie utilizes an Extended Kalman Filter. The information flow from component to component at low level can be seen below;
Recommendations and Troubleshooting
- It is vital to have sensors operating in proper environmental conditions. Such as non-reflective floor surface for (Flowdeck) laser ranger, textured surface for (Flowdeck) camera/ laser ranger, empty room for clear radio signalling to (LPS) UWBs.
- For packaging purposes of the battery, Flowdeck and LPS deck it is better to use long pins such that proper power and signal distribution is supported. If not, the contact pins for mounting these devices becomes only marginally sufficient and at some points will not power the devices properly.
- Stable flight can be achieved if at least 4 anchors are identified so to make sure there is sufficient coverage over the (Tech United) field it is best to use 8 anchors. With this set up there is a better chance that range data will be continuously pinged from the Crazyflie 2.0 deck and back.
- LPS and Flowdeck together when used especially with TWR mode the Loco deck crashes after a while. The issue also raised in ticket and another. If encountered, Issue #368 re-flash the drone. Link does not work.
- While testing it has sometimes been observed that beacons on ground level do not communicate with Client at start of connection. A ticket has been opened concluding some alternative solutions in the forum.
- Some parameter setting could be changed to test in the CF Firmware in cases which EKF is getting confused with conflicting data from flow and the LPS at the same time. Things to try could be to increase the standard deviation: For Flow sensor, For Range sensor, For UWB data parameter
Path Planning
Algorithm 1
Assumptions
Solution
Algorithm 2
The objective of this path planning algorithm is to find the shortest path for the drone while keeping the ball in the field of view. This is to minimise the mobility of the drone under given performance constraints to ensure a stable view from the onboard camera. This is done by keeping the ball's position as reference and thereafter, using the ball velocity to predict the future ball position.
Assumptions
- The altitude, $z_{drone}$ of the drone is same throughout the duration of the game.
- The ball is always on the ground, i.e., $z_{ball} = 0$.
- The drone encounters no obstacles at altitude, $z_{drone}$.
Solution
Simulation
Need introduction here.
Previous Game Data Extraction
To demonstrate the performance of our system, we planned to fly the drone both in the virtual world and the physical world. The virtual drone is for demonstrating that the output from our path planner is practical, and the physical drone is for demonstrating that our capability of flying the drone autonomously according to the output of the path planner.
To describe the whole scenario in detail. In the virtual world, a previous game of Tech United will be played, and the drone will fly around the field according to the output from the path planner. On the monitor, what the camera on the drone is filming will be shown. On the other hand, in the physical world, a drone will also be flying according to the command from the path planner, but there will be no game playing.
To realize the above-mentioned scenario, we need to extract the ball position from previous game data and transfer it to the path planner. In the beginning, we tried to retrieve the ball position from the data file loaded when replaying the previous game. After a few attempts, it turns out that it would be more straightforward to save the ball position into a matrix when it is used in the original program. A function is built to implement this.
Drone Camera View Control
To show what the camera is supposed to be filming in the virtual world, we need the position and yaw angle of the drone, and the pitch angle of the camera when it is mounted on the drone. With these defined inputs, we built a function which will read those data from a CSV format file, and change the displayed view in the simulator accordingly.
Perception
The fundamental goal is being able to referee the game of football. While testing for this project is a two vs. two game between two robots at the Tech United pitch facilities, the ultimate vision is that the system could be scaled and adapted for use in a real outdoor game between human beings. From the experience of previous projects, the client imposed the direction that the developed system uses drones with cameras. The main benefit of this being the relative convenience of setting up and the assumption that this dynamic capability would save cost by allowing for fewer cameras for comparable camera coverage.
The work activity relating to the camera system was divided into two workstreams to allow for concurrent work activity. Firstly, the selection and integration of the hardware, such that it would not adversely affect the flight dynamics of the drone. Owing to the small size of the Crazyflie 2 drone, and the additional safety issues with using the larger drone variant with extra load capacity, we were restricted to using a small FPS camera because of latency issues with other options, as discussed in Yussuf's section. While this option allows for a live camera feed to an external referee, this option seemingly does not allow for further processing of those images without significant delay. So this option would only ever allow for the most basic of systems, requiring humans firmly in the driving seat of all decision making. While this simplification was deemed an acceptable compromise for the scope of this project, it is assumed that the project will be continued by others, and so this is an essential factor to consider. In this project, we also assumed that the position of the ball and players is known, as this data has been taken from previous games that have been recorded from the server. In the real case, this would not be available, and mechanisms to determine this input must be a part of the system itself. Intending to address both of those points, the second workstream relating to the camera system focused on obtaining information from the data captured in the video streams. As the first step in that direction, the specific objective here was focused on getting the ball position so that this could input into Song and Ankita's control software and also update the simulator views. At this stage, the player positions have not been considered, as this information is required for more granular decisions.
Workstream 1
Workstream 2
Integration
Path Planning and Crazyflie
Path Planning and Simulator
Testing
The computed path planning algorithms are tested on the Visualiser and the hardware itself as explained as follows: \vspace{2mm}
- For a given recorded game, the Simulator is capable of providing the ball coordinates which are then fed into the path planning algorithm. The python script for both the algorithms generate a .CSV file with information on drone's reference position along with the yaw angle of the drone. This is sufficient information to set a frame of view within the game and visualise the activity of the active players and the ball. More information on the implementation of this method can be found here.
- After the successful implementation of the algorithm on the simulator, the reference drone position points are plugged into AutonomousSequence.py python script. These points are fed into the drone to have the ball in the field of view of drone's camera.
In a real test case scenario, the drone gets the information of the ball's position using the onboard camera and objective of the drone would then be to keep the ball in its field of view.