Embedded Motion Control 2014 Group 3: Difference between revisions
No edit summary |
mNo edit summary |
||
| Line 50: | Line 50: | ||
The following steps describe the algorithm to find the arrow and determine the direction: | The following steps describe the algorithm to find the arrow and determine the direction: | ||
'''1 Read rgb image from "/pico/asusxtion/rgb/image_color" topic.''' | '''1. Read rgb image from "/pico/asusxtion/rgb/image_color" topic.''' | ||
'''2 Convert rgb image to hsv color space.''' | '''2. Convert rgb image to hsv color space.''' | ||
[[File:Rgbtohsv1.png | thumb | Rgb to hsv color space]] | [[File:Rgbtohsv1.png | thumb | 2. Rgb to hsv color space]] | ||
'''3 Filter out the red color using cv::inRange''' | '''3. Filter out the red color using cv::inRange''' | ||
[[File:inrange.png | thumb | Filter out red color]] | [[File:inrange.png | thumb | 3. Filter out red color]] | ||
'''4 Find contours and convex hulls and filter it''' | '''4. Find contours and convex hulls and filter it''' | ||
The filter removes all contours where the following relationship does not hold: <math>0.5 < \frac{Contour \ area}{Convex \ hull \ area} < 0.65 </math>. This removes some of the unwanted contours. The contour and convex hull of the arrow: | The filter removes all contours where the following relationship does not hold: <math>0.5 < \frac{Contour \ area}{Convex \ hull \ area} < 0.65 </math>. This removes some of the unwanted contours. The contour and convex hull of the arrow: | ||
[[File:convex.png | thumb | Find contours]] | [[File:convex.png | thumb | 4. Find contours]] | ||
'''5 Use cv::approxPolyDP over the contours''' | '''5. Use cv::approxPolyDP over the contours''' | ||
The function cv::approxPolyDP is used to fit polylines over the resulting contours. The arrow should have approximately 7 lines per polyline. The polylines fitted over the contours with <math>5 \ < \ number \ of \ lines \ in \ polyline \ < \ 10 </math> is the arrow candidate. | The function cv::approxPolyDP is used to fit polylines over the resulting contours. The arrow should have approximately 7 lines per polyline. The polylines fitted over the contours with <math>5 \ < \ number \ of \ lines \ in \ polyline \ < \ 10 </math> is the arrow candidate. | ||
[[File:polylines.png | thumb | Fit polylines]] | [[File:polylines.png | thumb | 5. Fit polylines]] | ||
'''6 Determine if arrow is pointing left or right''' | '''6. Determine if arrow is pointing left or right''' | ||
First the midpoint of the arrow is found using <math>x_{mid} = \frac{x_{min}+x_{max}}{2}</math>. When the midpoint is known the program iterates over all points of the arrow contour. Two counters are made which count the number of points left and right of <math>x_{mid}</math>. If the left counter is greater than the right counter the arrow is pointing to the left, otherwise the arrow is pointing to the right. | First the midpoint of the arrow is found using <math>x_{mid} = \frac{x_{min}+x_{max}}{2}</math>. When the midpoint is known the program iterates over all points of the arrow contour. Two counters are made which count the number of points left and right of <math>x_{mid}</math>. If the left counter is greater than the right counter the arrow is pointing to the left, otherwise the arrow is pointing to the right. | ||
'''7 Making the detection more robust''' | '''7. Making the detection more robust''' | ||
As last an effort is made to make the arrow detection more robust, for example when at one frame the arrow is not detected the program still knows there is an arrow. This is done by taking the last 5 iterations, check if in all these iterations the arrow is detected then publish the direction of the arrow onto the topic "/arrow". If in the last 5 iterations no arrow is seen the arrow is not visible anymore thus publish that there is no arrow onto the topic "/arrow". | As last an effort is made to make the arrow detection more robust, for example when at one frame the arrow is not detected the program still knows there is an arrow. This is done by taking the last 5 iterations, check if in all these iterations the arrow is detected then publish the direction of the arrow onto the topic "/arrow". If in the last 5 iterations no arrow is seen the arrow is not visible anymore thus publish that there is no arrow onto the topic "/arrow". | ||
Revision as of 21:52, 10 June 2014
Group Members
| Name: | Student id: |
| Jan Romme | 0755197 |
| Freek Ramp | 0663262 |
| Anne Krus | 0734280 |
| Kushagra | 0873174 |
| Roel Smallegoor | 0753385 |
| Janno Lunenburg - Tutor | - |
Final software
Overview of final strategy
LaserProcessing
LaserData from Pico
The data from the laser on pico is in lasercloud format. This means that the data is represented in an array of distances. The starting angle and angle increment are known. This means we have the distances from laser to objects for a range of angles.
LaserCloud to Pointcloud
Because we are going to fit lines through the walls, it would be easier to have the data in Carthesian Coordinates. In this node the laserData is transformed into a PointCloud, which is published on the topic. It is also possible to filter the data in this node when needed. For now all data is transformed into the PointCloud.
Arrow detection
The following steps describe the algorithm to find the arrow and determine the direction:
1. Read rgb image from "/pico/asusxtion/rgb/image_color" topic.
2. Convert rgb image to hsv color space.
3. Filter out the red color using cv::inRange
4. Find contours and convex hulls and filter it
The filter removes all contours where the following relationship does not hold: [math]\displaystyle{ 0.5 \lt \frac{Contour \ area}{Convex \ hull \ area} \lt 0.65 }[/math]. This removes some of the unwanted contours. The contour and convex hull of the arrow:
5. Use cv::approxPolyDP over the contours
The function cv::approxPolyDP is used to fit polylines over the resulting contours. The arrow should have approximately 7 lines per polyline. The polylines fitted over the contours with [math]\displaystyle{ 5 \ \lt \ number \ of \ lines \ in \ polyline \ \lt \ 10 }[/math] is the arrow candidate.
6. Determine if arrow is pointing left or right
First the midpoint of the arrow is found using [math]\displaystyle{ x_{mid} = \frac{x_{min}+x_{max}}{2} }[/math]. When the midpoint is known the program iterates over all points of the arrow contour. Two counters are made which count the number of points left and right of [math]\displaystyle{ x_{mid} }[/math]. If the left counter is greater than the right counter the arrow is pointing to the left, otherwise the arrow is pointing to the right.
7. Making the detection more robust As last an effort is made to make the arrow detection more robust, for example when at one frame the arrow is not detected the program still knows there is an arrow. This is done by taking the last 5 iterations, check if in all these iterations the arrow is detected then publish the direction of the arrow onto the topic "/arrow". If in the last 5 iterations no arrow is seen the arrow is not visible anymore thus publish that there is no arrow onto the topic "/arrow".
PFM
Organization
Time survey
Link: time survey group 3
Planning
Week 1 (28/4 - 4/5)
Finish the tutorials
Week 2 (5/5 - 11/5)
Be able to detect walls and convert them to start and end points
Week 3 (12/5 - 18/5)
Finish strategy to be able to successfully finish the competition
Older Software
Overview of first strategy
In the overview the different packages (dotted boxes) and nodes (solid boxes) are displayed. The topics are displayed at the sides of the nodes.
Initial (boolean) safety
The safety node is created for testing. When something goes wrong and Pico is about to hit the wall the safety node will publish a Bool to tell the strategy it is not safe anymore. When the code is working well safety shouldn't be needed anymore.
Obstacle Detection
Finding Walls from PointCloud data
The node findWalls reads topic "/cloud" which contains laserdata in x-y coordinates relative to the robot. The node findWalls returns a list containing(xstart,ystart) and (xend, yend) of each found wall (relative to the robot). The following algorithm is made:
- Create a cv::Mat object and draw cv::circle on the cv::Mat structure corresponding to the x and y coordinates of the laserspots.
- Apply Probabilistic Hough Line Transform cv::HoughLinesP
- Store found lines in list and publish this on topic "/walls"
A visualization of the output (left: laserdata from the real Pico right: detected lines 'walls'):
Select Walls
In FindWalls lines are fitted over all the walls. In selectwalls the walls are filtered to find the two walls in the driving direction. The walls are send as a starting and endpoint. To be able to compare the walls to each other, the begin point is projected on x=0 (at height of Pico). The closest walls left and right of Pico with the same direction are the two walls to use for navigation.
Next part should be in strategy i think
From the grade of the walls compared to Pico a setpoint can be set on a certain distance in x direction, on which pico can correct to drive straight. A setpoint further away leads to smaller corrections compared to a setpoint closer to pico. The setpoint can also be uses for taking the turns.
Overview of second strategy
See diagram, circles represent separate nodes that communicate using topics. Blocks represent functions, which will be written in separate c++ files and included in the "main" c++ file. This way, no one will interfere in code of someone else.
Notes (TODO)
Week3
Combine detection and strategy part
+ Determine waypoint (turnpoint)
- Find door (using end of line)
+ Turn
+ Drive out of maze
Week 4 (week after corridor challenge)
Because of the Elevator project Freek and Anne are not responsible for anything this week.
Jan and Roel will write (optimize) the wall finding algorithm and use it also for the doors. Kushagra will write the safety node.