Autonomous Referee System: Difference between revisions

From Control Systems Technology Group
Jump to navigation Jump to search
No edit summary
No edit summary
 
(107 intermediate revisions by 4 users not shown)
Line 1: Line 1:
<div align="left">
<div align="left">
<font size="5">Autonomous Referee System</font><br />
<font size="4">'An objective referee for robot football'</font>
<font size="4">'An objective referee for robot soccer'</font>
</div>
</div>


=Introduction=
<div STYLE="float: left; width:80%">
<p>
</div><div style="width: 35%; float: right;"><center>{{:Content_MSD16_large}}</center></div>
__NOTOC__
 
 
 
A football referee can hardly ever make "the correct decision", at least not in the eyes of the thousands or sometimes millions of fans watching the game. When a decision will benefit one team, there will always be complaints from the other side. It is oft-times forgotten that the referee is also merely a human. To make the game more fair, the use of technology to support the referee is increasing. Nowadays, several stadiums are already equipped with [https://en.wikipedia.org/wiki/Goal-line_technology goal line technology] and referees can be assisted by a [http://quality.fifa.com/en/var/ Video Assistant Referee (VAR)]. If the use of technology keeps increasing, a human referee might one day become entirely obsolete. The proceedings of a match could be measured and evaluated by some system of sensors. With enough (correct) data, this system would be able to recognize certain events and make decisions based on these event.
 
 
The aim of this project is to do just that; making a system which can evaluate a soccer match, detect events and make decisions accordingly. Making a functioning system which could actually replace the human referee would probably take a couple of years, which we don't have. This project will focus on creating a high level system architecture and giving a prove of concept by refereeing a robot-soccer match, where currently the refereeing is also still done by a human. This project will build upon the [[Robotic_Drone_Referee|Robotic Drone Referee]] project executed by the first generation of Mechatronics System Design trainees.
 
 
To navigate through this wiki, the internal navigation box on the right side of the page can be used.
 
 
<center>[[File:tumbnail_test_video.png|center|750px|link=https://www.youtube.com/embed/XyRR3rPQ4R0?autoplay=1]]</center>
 
 
=Team=
This project was carried out for the second module of the 2016 MSD PDEng program. The team consisted of the following members:
This project was carried out for the second module of the 2016 MSD PDEng program. The team consisted of the following members:
* Farzad Mobini
* Akarsh Sinha
* Tuncay Olcer
* Farzad Mobini
* Jordy  Senden  
* Joep Wolken
* Tim Verdonschot
* Jordy  Senden
* Sa Wang
* Sa Wang
* Joep Wolken
* Tim Verdonschot
*     Akarsh Sinha
* Tuncay Uğurlu Ölçer
</p>
 
=Project Definition=
 
 
<center>[[File:Drone Ref.png|thumb|center|1000px|Illustration by Peter van Dooren, BSc student at Mechanical Engineering, TU Eindhoven, November 2016.]]</center>
 
=Acknowledgements=
A project like this is never done alone. We would like to express our gratitude to the following parties for their support and input to this project.
 
<center>[[File:logoAcknowledgements.png|center|1000px]]</center>
 
 
 
 
 
 
<!--
 
==Ground Robot==
 
[[File:Ground_Robot_specs.png|thumb|right|500px|Ground robot specs]]
 
[[File:Ground_Robot_overview.png|thumb|right|400px|Ground robot w.r.t. field]]
 
'''Requirements for Ground Robot'''
 
<br>
 
*''Motion:''
** The GR should be able to keep the ball in sight of its Kinect camera. If the ball is lost, GR should try to find it again with the Kinect.
** Since the ball is best tracked with the Kinect, the omni-vision camera can be used to keep track of the players.
 
<br>
 
*''Vision:''
** Position self with respect to field lines
** Detect ball
** Estimate global ball position and velocity
** Detect objects (players) in field
** Estimate global position and velocity of objects
** Determine which team the player belongs to
 
<br>
 
*''Communication:''
: Send to laptop:
:* Ball position + velocity estimate
:* Player position + velocity estimate
:* Player team/label
:* Own position + velocity
:* Own side/home goal
:* Own detection of B.O.O.P. or Collision (maybe)
 
: Receive from laptop:
:* Reference position
:* Detection flag
 
<br>
 
*''Extra:''
** Get ball after B.O.O.P.
** Communicate with second Ground Robot
 
==Drone==
*AR Parrot Drone Elite Addition 2.0
*19 min. flight time (ext. battery)
*720p Camera (but used as 360p)
*~70° Diagonal FOV (measured)
*Image ratio 16:9
===Drone control===
*Has own software & controller
*Possible to drive by MATLAB using arrow keys
*Driving via position command and format of the input data is a work to do
*x, y, θ position feedback via top cam and/or UWBS
*z position will be constant and decided according FOV
 
==Positioning==
 
Positioning System block is responsible for creating the reference position of the drone and the ground robot referee based on the information of the players and the ball. The low level controller of the both system will incorporate the reference position as a desired state for tracking purposes.
[[File:Positioning.png|thumb|right|400px|Depiction of the positioning subsystem.]]
Currently :
*Ground referee (Turtle) focuses on ball
*Drone focuses on collision/players
 
==Detection==
The fault detection should
*Receive images and estimations of state related parameter from the drone and the ground robot.
*Based on the information, evaluate which of the two rules (BOOP and Collision) are violated.
*Communicate with respective refs the final verdict
** Collaboration with the ground ref
*** Receive estimated
**** Ball Position and velocity 
**** Player position and velocity
**** Position of line/ ball boundary
*** Transmit decision flag regarding BOOP
** Collaboration with the drone ref
*** Receive estimated
**** Player position and velocity 
**** Ball Position and velocity 
*** Transmit decision flag regarding Collision 
 
<p>
<p>
As described in <ref>D. Antunes and R. Molengraft, Drone Referee, Control Systems Technology group, Mechanical Engineering Department,
===Definition of fault/foul===
TU Eindhoven, November 2016.</ref> verbatim, the goal of the present project is to contribute to this vision and create an autonomous robot referee system using drones. The first generation of MSD PDEng students created a system architecture <ref> [http://cstwiki.wtb.tue.nl/index.php?title=Robotic_Drone_Referee "Robotic Drone Referee"] </ref> to be used with a single drone. This architecture provides the basis for the present project. In particular some of the modules of such architecture, such as out of bound ball detection and an indoor positioning system using ultra-wind band technology, were implemented and tested. The overall goal of this project is to extend this system architecture and implement more modules.
The definition of foul/fault or offence is based on the Robo Cup MSL Rule Book <ref> [http://wiki.robocup.org/Middle_Size_League#Rules "Middle Size Robot League Rules and Regulations"] </ref> . Simple physical contact does not represent an offence. Speed and impact of physical contact shall be used to define offence or a foul. There are two cases in which foul detection should be formulated.
</p>
*'''Case 1: One of the robots is in possession of the ball'''
=Background=
[[File:Contact Between Robots.png|thumb|right|450px|Indirect (left) and direct (right) contact between robots. ]]
<p>
** A foul will be defined in this case if Robot B impedes the progress of the opponent by
A drone referee may provide several advantages with respect to a human referee or a camera based system covering the entire field. First, human referees, naturally prone to human errors, are one the main causes of controversy in the game; they have their own interpretation of the rules, introducing a non-predictable factor often leading to unfair situations in a game where both financial and emotion stakes are high. An autonomous system would mitigate this, and in particular remove the unfairness factor - every game would be refereed according to the same algorithm.
**#Colliding after charging at A with  v  unit velocity
**#Applying (instantaneous) pushing with ≥ 𝑭  unit force
**#Continuing to push for time ≥ t seconds
**#Knocking the ball off A by sudden (Instantaneous)  application of force (≥ 𝑭 unit force)
*Possible ways of measuring these
***Velocity
**#Visual odometry (Image-based Object Velocity Estimation)
***Application of (instantaneous) force
**#Use visual odometry and calculate velocity/ acceleration and include time data.  
**#Estimate force accordingly
**Continuous push (B is pushing A)
**#Detect instantaneous application of F unit force
**#Detect if B changes direction of movement within  t seconds
**Knocking off ball (only visual data)
**#Detect collision
**#Detect ball and Player A after collision
 
*'''Case 2: None of the robots are in possession of the ball'''
[[File:No Robot Has Ball Possession.png|thumb|right|300px|No robot has ball possession.]]
**A foul will be defined in this case if Robot either A or B impedes the progress of the opponent by
**#Colliding with larger momentum (say, pB ≥ pA  units)
**#Continues with the momentum the  for time ≥ t seconds (dp/dt=0,for t seconds after impact)
**Possible ways of measuring these
***Momentum
***#Use visual odometry to estimate velocity (and elapsed time)
***#Estimate momentum accordingly
***Continuous application of momentum
***#Detect if defaulter changes direction of movement within t seconds
</p>
</p>


=Project Objectives=
==Image processing==
<p>
===Capturing images===
* System architecture of the proposed solution by January 31st along with a time plan, risk assessment of the choices, and task distribution for the elements of the group.
'''Objective''': Capturing images from the (front) camera of the drone.
* Software of the proposed solutions including:
 
** Out of bound ball detection by the ground robot, including both motion algorithm and camera processing. Suggested: end of January.
 
** Detection of a fault including both movement. Suggested: end of February.
'''Method''':
*MATLAB
** ffmpeg
** ipcam
** gigecam
** hebicam
* C/C++/Java/Python
** opencv
…
No method chosen yet, but ipcam, gigecam and hebicam are tested and do not work for the camera of the drone. FFmpeg is also tested and does work, but capturing one image takes 2.2s which is way too slow. Therefore, it might be better to use software written in C/C++ instead of MATLAB.
 
===Processing images===
'''Objective''': Estimating the player (and ball?) positions from the captured images.
 
 
'''Method''': Detect ball position (if on the image) based on its (orange/yellow) color and detect the player positions based on its shape/color (?).
 
== Top Camera ==
The topcam is a camera that is fixed above the playing field. This camera is used to estimate the location and orientation of the drone. This estimation is used as feedback for the drone to position itself to a desired location.
 


* Software with the interaction between the two robots. Suggested: end of March.
The topcam can stream images with a framerate of 30 Hz to the laptop, but searching the image for the drone (i.e. image processing) might be slower. This is not a problem, since the positioning of the drone itself is far from perfect and not critical as well. As long as the target of interest (ball, players) is within the field of view of the drone, it is acceptable.
* Demo to be scheduled by the end of March or beginning of April.
* A Wiki-page documenting the project and providing a repository for the software developed, similar to the one obtained from the first generation of MSD students.
* One minute long video to be used in presentations illustrating the work.


</p>


==References==
=References=
<references/>
<references/>
-->

Latest revision as of 16:07, 24 October 2017

'An objective referee for robot football'



A football referee can hardly ever make "the correct decision", at least not in the eyes of the thousands or sometimes millions of fans watching the game. When a decision will benefit one team, there will always be complaints from the other side. It is oft-times forgotten that the referee is also merely a human. To make the game more fair, the use of technology to support the referee is increasing. Nowadays, several stadiums are already equipped with goal line technology and referees can be assisted by a Video Assistant Referee (VAR). If the use of technology keeps increasing, a human referee might one day become entirely obsolete. The proceedings of a match could be measured and evaluated by some system of sensors. With enough (correct) data, this system would be able to recognize certain events and make decisions based on these event.


The aim of this project is to do just that; making a system which can evaluate a soccer match, detect events and make decisions accordingly. Making a functioning system which could actually replace the human referee would probably take a couple of years, which we don't have. This project will focus on creating a high level system architecture and giving a prove of concept by refereeing a robot-soccer match, where currently the refereeing is also still done by a human. This project will build upon the Robotic Drone Referee project executed by the first generation of Mechatronics System Design trainees.


To navigate through this wiki, the internal navigation box on the right side of the page can be used.


Tumbnail test video.png


Team

This project was carried out for the second module of the 2016 MSD PDEng program. The team consisted of the following members:

  • Akarsh Sinha
  • Farzad Mobini
  • Joep Wolken
  • Jordy Senden
  • Sa Wang
  • Tim Verdonschot
  • Tuncay Uğurlu Ölçer


Illustration by Peter van Dooren, BSc student at Mechanical Engineering, TU Eindhoven, November 2016.

Acknowledgements

A project like this is never done alone. We would like to express our gratitude to the following parties for their support and input to this project.

LogoAcknowledgements.png




Retrieved from "https://dsdwiki.wtb.tue.nl/index.php?title=Autonomous_Referee_System&oldid=45730"