AutoRef - Autonomous Referee System: Difference between revisions

From Control Systems Technology Group
Jump to navigation Jump to search
No edit summary
mNo edit summary
 
(199 intermediate revisions by 2 users not shown)
Line 1: Line 1:
<span style="color:red">
[[File:Autoref_title_logo_v2.jpg|thumb|right|128px|AutoRef logo]]
''Note: This page is being actively modified as of 30 March 2021.''
</span>


The '''AutoRef''' system is a proposed fully autonomous referee for [https://msl.robocup.org/ RoboCop Middle Size League] robot soccer under development as a continuous project at Eindhoven University of Technology (TU/e). Development of the system architecture and implementation began in 2016 by TU/e PDEng trainees in the 2015–2017 cohort of the [https://www.tue.nl/studeren/graduate-school/pdeng-automotive-systems-design/about-msd/ Mechatronic Systems Design (MSD)] program as their Block II in-house project; since then, each MSD cohort has contributed to AutoRef's continuation in their respective Block II in-house projects. In 2019, a team of students from the TU/e Honors program [https://educationguide.tue.nl/programs/tue-honors-academy/bachelor-students/honors-tracks/high-tech-systems/?L=2 High Tech Systems] realized the control of drones as part of AutoRef's implementation, with continued efforts in drone control by these students leading to the [https://fireflyeindhoven.nl/ Firefly-project].
The '''AutoRef''' system is a proposed autonomous referee for [https://msl.robocup.org/ RoboCup Middle Size League (MSL) robot soccer] under development at Eindhoven University of Technology (TU/e). It is intended to accomplish all refereeing duties of human referees in MSL.


AutoRef has experienced multiple iterations of its project definition, consequently leading to poor progress between different teams. While earlier MSD cohorts emphasized a fully autonomous referee, later years scoped the project towards an assistive system for human referees to use for complex refereeing duties. To prevent further repetitive efforts in the design of AutoRef, the 2020–2022 MSD cohort (''AutoRef MSD 2020'' or simply ''MSD 2020'') was specifically tasked with ensuring continuity for future work. Therefore, as of MSD 2020, the AutoRef's system architecture and implementation pages describe the current state of the project "as-is" without the voice or perspective of any specific team's contribution. Contributions from previous teams (i.e., MSD 2015–2019 and Honors 2019) are described on their own respective pages, with an archive provided by MSD 2020 to systematically describe team contributions {{:Archive hyperlink needed}}.
Development on '''[[AutoRef system architecture|AutoRef's system architecture]]''' and '''[[AutoRef implementation|implementation]]''' began in 2016 by TU/e PDEng trainees in the 2015–2017 cohort of the [https://www.tue.nl/studeren/graduate-school/pdeng-automotive-systems-design/about-msd/ Mechatronic Systems Design (MSD)] program as their Block II in-house project. Since then, each MSD cohort team has contributed to AutoRef's continuation in their respective Block II in-house projects. In TU/e's 2019–2020 academic year, a team of students in the Honors program [https://educationguide.tue.nl/programs/tue-honors-academy/bachelor-students/honors-tracks/high-tech-systems/?L=2 High Tech Systems] realized the control of drones as part of AutoRef's implementation, with continued efforts in drone control by these students leading to the [https://fireflyeindhoven.nl/ Firefly-project].
 
[[#Team contributions|Team contributions]] to AutoRef from 2016 through 2020 saw multiple iterations of its project definition, consequently leading to relatively poor overall project progress. Earlier MSD cohorts emphasized a fully autonomous referee, while later years scoped the project towards an assistive, supplementary system for human referees. In 2021, to prevent further repetitive efforts in the design of AutoRef, the 2020–2022 MSD cohort project team (''AutoRef PDEng MSD 2020'' or simply ''MSD 2020'') was specifically tasked with ensuring continuity for future work.
 
AutoRef's wiki pages (including the [[AutoRef - Autonomous Referee System|main]], [[AutoRef system architecture|system architecture]], and [[AutoRef implementation|implementation]] pages) describe the current state of the project as a collective of team contributions starting from MSD 2020. [[#Team contributions|Team contributions]] prior to MSD 2020 (i.e., MSD 2015–2019 and Honors 2019) are generally provided as isolated technical reports which provide details of their respective work but do not meet the concern for continuity of AutoRef. Contributions from MSD 2020 onwards are integrated into the AutoRef's pages for continuity, with each team's respective contribution page providing summaries of what was done to help supervisors understand and evaluate a team's work. An archive providing an overview of all team contributions is [https://tuenl.sharepoint.com/sites/MCS_Drone_Referee_project_Team/_layouts/15/Doc.aspx?OR=teams&action=edit&sourcedoc={EA8BF37F-3469-4424-9010-BEADD8E96EE9} downloadable with access to AutoRef's TU/e SharePoint folder].
__TOC__
__TOC__


==Project origins==
==Background==
:''This section's tone or style may not reflect the encyclopedic tone as used in articles such as those on Wikipedia. See Wikipedia's [https://en.wikipedia.org/wiki/Wikipedia:Writing_better_articles#Tone guide to writing better articles for suggestions].''
 
===RoboCup===
===RoboCup===
A football referee can hardly ever make "the correct decision", at least not in the eyes of the thousands or sometimes millions of fans watching the game. When a decision will benefit one team, there will always be complaints from the other side. It is oft-times forgotten that the referee is also merely a human. To make the game more fair, the use of technology to support the referee is increasing. Nowadays, several stadiums are already equipped with [https://en.wikipedia.org/wiki/Goal-line_technology goal line technology] and referees can be assisted by a [http://quality.fifa.com/en/var/ Video Assistant Referee (VAR)]. If the use of technology keeps increasing, a human referee might one day become entirely obsolete. The proceedings of a match could be measured and evaluated by some system of sensors. With enough (correct) data, this system would be able to recognize certain events and make decisions based on these event.
[http://www.robocup.org/ RoboCup] is an international initiative to promote and advance research in robotics and artificial intelligence. Founded in 1997, its main goal is to ‘develop a team of fully autonomous humanoid robot soccer players that is able to win against the winner of the most recent World Cup, complying with the official rules of FIFA, by the middle of the 21st century’.
 
====Middle Size League (MSL)====
In RoboCup's [https://msl.robocup.org/ Middle Size League (MSL)], two teams of five autonomous robots play a soccer match on an artificial field. These robots are able to drive around while using several on-board camera's to position themselves on the field. Moreover, they can determine the position of the ball, opponents and team mates. Through radio signals they can communicate with each other and decide upon a strategy. With a ball-handling system the ball can be captured and controlled and a shooting mechanism is able to shoot a ball over the ground or through the air.
 
As discussed in this [http://wiki.robocup.org/images/7/72/RequirementsforMSL_2016.pdf tutorial], a standard RoboCup field measures 18 by 12 meters. During a match, there are two teams consisting of five robots on this field, driving around with velocities up to 5 m/s and possibly even higher. These robots are all competing for the same thing: scoring goals. This means that getting possession of the ball is a primary goal. When several robots are competing for the ball, collisions, pushing and scrummages are nearly inevitable.
 
====Human refereeing====
To ensure each MSL match is played fairly, a human referee observes events on the field from the sideline. This human referee is supported by an auxiliary referee who stands on the opposite side, next to the field. Both referees can stop the game in response to a committed foul, a scored goal, a ball out of bounds, and other game events as specified in the MSL rules. The [https://msl.robocup.org/rules MSL rules] are based on the official FIFA rules, but adapted to robot soccer rules were necessary. However, the large set of rules and the interpretation thereof can often lead to situations where a referee might decide to continue the game, while another might decide to interrupt. This can and will often lead to frustrations in the aggrieved team. Moreover, a decision made by a referee can affect the outcome of a game and even an entire championship.


====RoboCup 2016 Leipzig: MSL final match====
The outcome-affecting nature of refereeing was evident in the final match of the [http://www.robocup2016.org/en/ RoboCup world championship 2016] in Leipzig, Germany ([https://www.youtube.com/watch?v=f7Y6QLYVhSs&feature=youtu.be&t=6h17m38s full match]; [https://www.youtube.com/watch?v=2JxNjgKE8HQ match highlights]). The final was played between team [http://www.techunited.nl/ Tech United] from the Netherlands and team [http://blog.sina.com.cn/s/articlelist_2532664717_0_1.html%E2%80%8D WATER] from China. The winner of this would become world champion robot soccer in the MSL. At the end of the match, the scoreboard showed 2-2. Per MSL rules (as is in FIFA rules), a tied score results in extra time to decide on the winner. During the match, team WATER faced issues with ball handling, preventing the ball from rotating in a ‘natural’ way over the field. When it happens that the ball does not rotate in the direction it is being moved, this is considered clamping and regarded as a foul in favor of the other team. In the last couple of minutes the score was 3-3 when WATER turned towards the Tech United goal, shot and scored the winning goal. While the Chinese team was already celebrating their victory, the auxiliary referee decided that the scoring robot was clamping the ball before scoring the goal. After a discussion with the main referee, it was decided to declare the goal invalid. Since the extra time also ended in a draw, penalties were needed to decide who would become the new world champion. After all penalties of the Chinese team were stopped by the Dutch keeper, the first shot of the Tech United robot went into the net. The Dutch team won the penalty series with 1-0 and thus Tech United became the world champion of 2016.


The aim of this project is to do just that:
<center>[[File:tumbnail_test_video.png|center|750px|link=https://www.youtube.com/embed/XyRR3rPQ4R0?autoplay=1]]</center>
<br>'''''Creating a system which can evaluate a soccer match, detect events and make decisions accordingly.'''''


===AutoRef motivation===
===AutoRef motivation===


<p>[http://www.robocup.org/ RoboCup] is an international initiative to promote and advance research in robotics and  artificial intelligence. Founded in 1997, its main goal is to ‘develop a team of fully autonomous humanoid robot soccer players that is able to win against the winner of the most recent World Cup, complying with the official rules of FIFA, by the middle of the 21st century’. In the Middle Size League ([http://wiki.robocup.org MSL]), two teams of five autonomous robots play a soccer match on an artificial field. These robots are able to drive around while using several on-board camera's to position themselves on the field. Moreover, they can determine the position of the ball, opponents and team mates. Through radio signals they can communicate with each other and decide upon a strategy. With a ball-handling system the ball can be captured and controlled and a shooting mechanism is able to shoot a ball over the ground or through the air.</p>
The events of the 2016 final match between the Dutch and Chinese MSL teams show how the decisions of human referees can affect the outcome of a match or even a tournament. Rules are always prone to interpretation, and a team which is disadvantaged by this will always complain. Referees have little means to justify their decisions other than their own observations and interpretations of the rules.
 
Referees in human soccer already use technology to support their decisions. Nowadays, several stadiums are already equipped with [https://en.wikipedia.org/wiki/Goal-line_technology goal line technology] and referees can be assisted by a [http://quality.fifa.com/en/var/ Video Assistant Referee (VAR)]. If the use of technology keeps increasing, a human referee for might one day become entirely obsolete. The proceedings of a match could be measured and evaluated by some system of sensors. With enough (correct) data, this system would be able to recognize certain events and make decisions based on these event.
 
Robot soccer in RoboCup MSL can likewise benefit from such an autonomous refereeing system — an ''AutoRef'' — to ensure matches are played fairly.
 
==Development history==
 
Development of the AutoRef [[AutoRef system architecture|system architecture]] and [[AutoRef implementation|implementation]] began in 2016 by TU/e PDEng trainees in the 2015–2017 cohort of the [https://www.tue.nl/studeren/graduate-school/pdeng-automotive-systems-design/about-msd/ Mechatronic Systems Design (MSD)] program as their Block II in-house project. Since then, each MSD cohort has contributed to AutoRef's continuation in their respective Block II in-house projects. In TU/e's 2019–2020 academic year, a team of students in the Honors program [https://educationguide.tue.nl/programs/tue-honors-academy/bachelor-students/honors-tracks/high-tech-systems/?L=2 High Tech Systems] realized the control of drones as part of AutoRef's implementation, with continued efforts in drone control by these students leading to the [https://fireflyeindhoven.nl/ Firefly-project].
 
[[#Team contributions|Team contributions]] in AutoRef's development history are divided by a paradigm shift starting with the 2020–2022 MSD cohort project team (''AutoRef PDEng MSD 2020'' or simply ''MSD 2020'') contribution in 2021. Unlike the teams which came before it, MSD 2020 eliminated the requirement of quadcopter drones as the basis for the autonomous refereeing system and emphasized continuity as a key stakeholder concern in system architecture. As such, the history of AutoRef is presented across:
#the [[#2015–2020 (drone-based system)|2015–2020 system]] based on various quadcopter drones; and
#the [[#2021–present|2021–present system]].
 
===2016–2020 (drone-based system)===
:''See also: [[#Team contributions|Team contributions]]
 
From 2016 through 2020 AutoRef's development was based on camera-equipped [https://en.wikipedia.org/wiki/Quadcopter quadcopter drones] in autonomously refereeing [https://msl.robocup.org/ RoboCup Middle Size League (MSL) robot soccer] matches. Most of the 2016–2020 projects introduced their own [https://en.wikipedia.org/wiki/Systems_engineering systems engineering] and implementation approaches. This repetitive work combined with the relatively short two-month duration available to these teams prevented the entire [https://en.wikipedia.org/wiki/V-Model V-model] from being realized for AutoRef and therefore causing relatively poor overall project progress across different teams.
 
Repetitions of the project definition (i.e., the left-hand side of the V-model) are specifically evident across the 2016–2020 team documentation. Earlier MSD cohorts emphasized a fully autonomous referee, while later years scoped the project towards an assistive, supplementary system for human referees. All teams from 2016–2020 specify the requirement that the AutoRef system be based on quadcopter drones. The majority of teams each used a different type of drone model in their implementation. This discontinuity in drone hardware between teams (among other hardware differences) further contributed to the slow progress AutoRef's development. To summarize the AutoRef 2016–2020 team contributions in terms of the underlying drone hardware:
*[[Robotic Drone Referee|MSD 2015]] n/a (specified a drone-based system, but did not implement a specific type of drone)
*[[Autonomous Referee System|MSD 2016]] Parrot AR.Drone 2.0 Elite Edition
*[[Drone Referee - MSD 2017/18|MSD 2017]] custom-built drone based on a Pixhawk PX4 controller
*[[Drone_Referee_-_MSD_2018/9|MSD 2018]] Avular Curiousity
*[[AutoRef_MSD_2019|MSD 2019]] Crazyflie 2.X
*[[AutoRef_honors_2019|Honors 2019]] Crazyflie 2.1
 
<center>[[File:Drone Ref.png|thumb|center|720px|Illustration by Peter van Dooren, BSc student at Mechanical Engineering, TU Eindhoven, November 2016.]]</center>


<p>As discussed in this [http://wiki.robocup.org/images/7/72/RequirementsforMSL_2016.pdf tutorial], a standard RoboCup field measures 18 by 12 meters. During a match, there are two teams consisting of five robots on this field, driving around with velocities up to 5 m/s and possibly even higher. These robots are all competing for the same thing: scoring goals. This means that getting possession of the ball is a primary goal. When several robots are competing for the ball, collisions, pushing and scrummages are nearly inevitable. To make sure the match is played in a fair way, a human referee keeps a keen eye on the events on the field from the sideline. This human referee is backed up by an auxiliary referee which is standing on the opposite side, next to the field. Both can decide on stopping the game, due to a committed foul, a scored goal, a ball out of bound or any other event. The [http://wiki.robocup.org/images/0/0f/Robocup-msl-rules-2016.pdf rules] for MSL are based on the official FIFA rules, but adapted to robot football rules were necessary. However, the large set of rules and the interpretation thereof can often lead to situations where a referee might decide to continue the game, while another might decide to interrupt. This can and will often lead to frustrations in the aggrieved team. Moreover, a decision made by a referee can affect the outcome of a game and even an entire championship.</p>
===2021–present===
:''See also: [[#Team contributions|Team contributions]]


<p>An example of this is the final match of the [http://www.robocup2016.org/en/ RoboCup world championship 2016] in Leipzig, Germany (see [https://www.youtube.com/watch?v=f7Y6QLYVhSs&feature=youtu.be&t=6h17m38s full match]/[https://www.youtube.com/watch?v=2JxNjgKE8HQ highlights]). The final was played between team [http://www.techunited.nl/ TechUnited] from the Netherlands and team [http://blog.sina.com.cn/s/articlelist_2532664717_0_1.html%E2%80%8D WATER] from China. The winner of this would become world champion robot soccer in the MSL. At the end of the match the scoreboard showed 2-2. As in human soccer, this means extra time to decide on the winner. During the match, team WATER had some trouble with the ball handling, preventing the ball to rotate in a ‘natural’ way over the field. When it happens that the ball does not rotate in the direction it is being moved, this is considered clamping and regarded as a foul in favor of the other team. In the last couple of minutes the score was 3-3 when WATER turned towards the TechUnited goal, shot and scored the winning goal. While the Chinese team was already celebrating their victory, the auxiliary referee decided that the scoring robot was clamping the ball before scoring the goal. After a discussion with the main referee, it was decided to declare the goal invalid. Since the extra time also ended in a draw, penalties were needed to decide who would become the new world champion. After all penalties of the Chinese team were stopped by the Dutch keeper, the first shot of the TechUnited robot went into the net. The Dutch team won the penalty series with 1-0 and thus TechUnited became the world champion of 2016.</p>
In 2021, to prevent further repetitive efforts and slowed progress in the development of AutoRef, the MSD 2020 team was tasked with ensuring continuity in their contribution to the project. AutoRef's technical specification resumed that of ''an autonomous refereeing system as to fully replace human refereeing in MSL''. The drone-based specification for AutoRef featured in previous team systems — that is, the requirement that AutoRef use drones — was also eliminated by MSD 2020.


<p>This example shows how important the decisions of the human referee team can be in shaping the course of a match or even a tournament. Rules are always prone to interpretation and a team which is disadvantaged by this will always complain. The referee has no means to justify his decision other than his own intuition and interpretation on the rules. This lead to the question on whether it would be possible to develop a system which can support the human referee team in making decisions. Such system might even become fully autonomous and could replace the human factor in refereeing entirely.</p>
As of MSD 2020, AutoRef's wiki pages (including the [[AutoRef - Autonomous Referee System|main]], [[AutoRef system architecture|system architecture]], and [[AutoRef implementation|implementation]] pages) describe the current state of the project as a collective of team contributions. Whereas pre-MSD 2020 team contribution pages provide standalone descriptions of their respective architecture and implemention, pages for MSD 2020 onwards only summarize their contributions, keeping the detailed documentation on AutoRef's pages. An archive was also initiated by MSD 2020 to satisfy their stakeholder requirement for an overview of all team contributions, which is [https://tuenl.sharepoint.com/sites/MCS_Drone_Referee_project_Team/_layouts/15/Doc.aspx?OR=teams&action=edit&sourcedoc={EA8BF37F-3469-4424-9010-BEADD8E96EE9} downloadable with access to AutoRef's TU/e SharePoint folder].


<center>[[File:tumbnail_test_video.png|center|750px|link=https://www.youtube.com/embed/XyRR3rPQ4R0?autoplay=1]]</center>
==System architecture==
:''Main article: [[AutoRef system architecture]]''
 
==Implementation==
:''Main article: [[AutoRef implementation]]''
 
==Team contributions==
 
Team contribution pages from 2016–2020 provide standalone descriptions of their respective architecture and implemention for AutoRef. Team contribution pages from 2021 onwards only summarize the team's respective contributions, keeping the detailed documentation on AutoRef's pages to ensure continuity. An archive provides a spreadsheet-based overview of all team contributions [https://tuenl.sharepoint.com/sites/MCS_Drone_Referee_project_Team/_layouts/15/Doc.aspx?OR=teams&action=edit&sourcedoc={EA8BF37F-3469-4424-9010-BEADD8E96EE9} downloadable with access to AutoRef's TU/e SharePoint folder].


==Development==
===Teams 2016–2020 (drone-based system)===


===History 2015–2020===
====[[AutoRef_Teams#PDEng_MSD|PDEng]]====
This project started in 2015 with the idea to use drones, equipped with cameras, to help monitor a [https://www.robocup.org/leagues/6 RoboCup] [https://www.robocup.org/leagues/6 Middle Size League (MSL)] soccer match. The use of drones was a first though towards achieving the goal. ''While drone can certainly play a part in the system, we must not limit ourselves to them.'' To better understand how the system should look like, a [https://en.wikipedia.org/wiki/Systems_engineering systems engineering] approach should be taken during the design. From the start, this project was part of the [https://www.tue.nl/studeren/alle-opleidingen/pdeng-opleidingen/ PDEng] [https://www.tue.nl/studeren/graduate-school/pdeng-automotive-systems-design/about-msd/ MSD] curriculum. The PDEng trainees were tasked with the development of the system architecture and implementation. Parallel to this, a team of students from the TU/e Honors programme [https://educationguide.tue.nl/programs/tue-honors-academy/bachelor-students/honors-tracks/high-tech-systems/?L=2 High Tech Systems] are tasked with the control of the drones. One of these teams have continued their efforts in drone-control by creating the [https://fireflyeindhoven.nl/ Firefly-project].
*[[Robotic Drone Referee|MSD 2015 (cohort 2015–2017)]]
*[[Autonomous Referee System|MSD 2016 (cohort 2016–2018)]]
*[[Drone Referee - MSD 2017/18|MSD 2017 (cohort 2017–2019)]]
*[[Drone_Referee_-_MSD_2018/9|MSD 2018 (cohort 2018–2020)]]
*[[AutoRef_MSD_2019|MSD 2019 (cohort 2019–2021)]]


The projects described are often of short duration. Moreover, every year the projects are done with new teams, which have their own ideas and approaches. These teams create their own documentation to show what they have done, such that they can be graded. Sometimes they use previous work as inspiration, but often they start from scratch with their own ideas. While this can give new perspectives on the system, it slows down the progress of creation. Instead of going progressing through the entire [https://en.wikipedia.org/wiki/V-Model V-Model] with each new group, we keep repeating the Project Definition (left side of the V).
====[[AutoRef_Teams#HTS_Honors|High Tech Systems (HTS)]]====
*[[AutoRef_honors_2019|Honors 2019 (academic year 2019–2020)]] with further drone work in [[Firefly_Eindhoven|Firefly project - Honors]]


This wiki-page is created in order to centralize the efforts made by several different students and teams. This page will act as the central backbone of the system. All decisions, requirements, functionalities etc. will be explained here. Since we are building ONE system, we should only need ONE backbone. This approach can lead to difficulties in grading the students/teams, since it might not be clear what part exactly they have contributed to. Therefore a separate page is made for each team. Here they can explain their contributions to the system (see navigation box).
===Teams 2021–present===


<center>[[File:Drone Ref.png|thumb|center|1000px|Illustration by Peter van Dooren, BSc student at Mechanical Engineering, TU Eindhoven, November 2016.]]</center>
====[[AutoRef_Teams#PDEng_MSD|PDEng]]====


===System architecture===
*[[AutoRef_MSD_2020|MSD 2020 (cohort 2020–2022)]]


===Implementation===
===Archive overview of team contributions===
[https://tuenl.sharepoint.com/sites/MCS_Drone_Referee_project_Team/_layouts/15/Doc.aspx?OR=teams&action=edit&sourcedoc={EA8BF37F-3469-4424-9010-BEADD8E96EE9} Downloadable with access to AutoRef's TU/e SharePoint folder].

Latest revision as of 09:28, 5 April 2021

AutoRef logo

The AutoRef system is a proposed autonomous referee for RoboCup Middle Size League (MSL) robot soccer under development at Eindhoven University of Technology (TU/e). It is intended to accomplish all refereeing duties of human referees in MSL.

Development on AutoRef's system architecture and implementation began in 2016 by TU/e PDEng trainees in the 2015–2017 cohort of the Mechatronic Systems Design (MSD) program as their Block II in-house project. Since then, each MSD cohort team has contributed to AutoRef's continuation in their respective Block II in-house projects. In TU/e's 2019–2020 academic year, a team of students in the Honors program High Tech Systems realized the control of drones as part of AutoRef's implementation, with continued efforts in drone control by these students leading to the Firefly-project.

Team contributions to AutoRef from 2016 through 2020 saw multiple iterations of its project definition, consequently leading to relatively poor overall project progress. Earlier MSD cohorts emphasized a fully autonomous referee, while later years scoped the project towards an assistive, supplementary system for human referees. In 2021, to prevent further repetitive efforts in the design of AutoRef, the 2020–2022 MSD cohort project team (AutoRef PDEng MSD 2020 or simply MSD 2020) was specifically tasked with ensuring continuity for future work.

AutoRef's wiki pages (including the main, system architecture, and implementation pages) describe the current state of the project as a collective of team contributions starting from MSD 2020. Team contributions prior to MSD 2020 (i.e., MSD 2015–2019 and Honors 2019) are generally provided as isolated technical reports which provide details of their respective work but do not meet the concern for continuity of AutoRef. Contributions from MSD 2020 onwards are integrated into the AutoRef's pages for continuity, with each team's respective contribution page providing summaries of what was done to help supervisors understand and evaluate a team's work. An archive providing an overview of all team contributions is downloadable with access to AutoRef's TU/e SharePoint folder.

Background

This section's tone or style may not reflect the encyclopedic tone as used in articles such as those on Wikipedia. See Wikipedia's guide to writing better articles for suggestions.

RoboCup

RoboCup is an international initiative to promote and advance research in robotics and artificial intelligence. Founded in 1997, its main goal is to ‘develop a team of fully autonomous humanoid robot soccer players that is able to win against the winner of the most recent World Cup, complying with the official rules of FIFA, by the middle of the 21st century’.

Middle Size League (MSL)

In RoboCup's Middle Size League (MSL), two teams of five autonomous robots play a soccer match on an artificial field. These robots are able to drive around while using several on-board camera's to position themselves on the field. Moreover, they can determine the position of the ball, opponents and team mates. Through radio signals they can communicate with each other and decide upon a strategy. With a ball-handling system the ball can be captured and controlled and a shooting mechanism is able to shoot a ball over the ground or through the air.

As discussed in this tutorial, a standard RoboCup field measures 18 by 12 meters. During a match, there are two teams consisting of five robots on this field, driving around with velocities up to 5 m/s and possibly even higher. These robots are all competing for the same thing: scoring goals. This means that getting possession of the ball is a primary goal. When several robots are competing for the ball, collisions, pushing and scrummages are nearly inevitable.

Human refereeing

To ensure each MSL match is played fairly, a human referee observes events on the field from the sideline. This human referee is supported by an auxiliary referee who stands on the opposite side, next to the field. Both referees can stop the game in response to a committed foul, a scored goal, a ball out of bounds, and other game events as specified in the MSL rules. The MSL rules are based on the official FIFA rules, but adapted to robot soccer rules were necessary. However, the large set of rules and the interpretation thereof can often lead to situations where a referee might decide to continue the game, while another might decide to interrupt. This can and will often lead to frustrations in the aggrieved team. Moreover, a decision made by a referee can affect the outcome of a game and even an entire championship.

RoboCup 2016 Leipzig: MSL final match

The outcome-affecting nature of refereeing was evident in the final match of the RoboCup world championship 2016 in Leipzig, Germany (full match; match highlights). The final was played between team Tech United from the Netherlands and team WATER from China. The winner of this would become world champion robot soccer in the MSL. At the end of the match, the scoreboard showed 2-2. Per MSL rules (as is in FIFA rules), a tied score results in extra time to decide on the winner. During the match, team WATER faced issues with ball handling, preventing the ball from rotating in a ‘natural’ way over the field. When it happens that the ball does not rotate in the direction it is being moved, this is considered clamping and regarded as a foul in favor of the other team. In the last couple of minutes the score was 3-3 when WATER turned towards the Tech United goal, shot and scored the winning goal. While the Chinese team was already celebrating their victory, the auxiliary referee decided that the scoring robot was clamping the ball before scoring the goal. After a discussion with the main referee, it was decided to declare the goal invalid. Since the extra time also ended in a draw, penalties were needed to decide who would become the new world champion. After all penalties of the Chinese team were stopped by the Dutch keeper, the first shot of the Tech United robot went into the net. The Dutch team won the penalty series with 1-0 and thus Tech United became the world champion of 2016.

Tumbnail test video.png

AutoRef motivation

The events of the 2016 final match between the Dutch and Chinese MSL teams show how the decisions of human referees can affect the outcome of a match or even a tournament. Rules are always prone to interpretation, and a team which is disadvantaged by this will always complain. Referees have little means to justify their decisions other than their own observations and interpretations of the rules.

Referees in human soccer already use technology to support their decisions. Nowadays, several stadiums are already equipped with goal line technology and referees can be assisted by a Video Assistant Referee (VAR). If the use of technology keeps increasing, a human referee for might one day become entirely obsolete. The proceedings of a match could be measured and evaluated by some system of sensors. With enough (correct) data, this system would be able to recognize certain events and make decisions based on these event.

Robot soccer in RoboCup MSL can likewise benefit from such an autonomous refereeing system — an AutoRef — to ensure matches are played fairly.

Development history

Development of the AutoRef system architecture and implementation began in 2016 by TU/e PDEng trainees in the 2015–2017 cohort of the Mechatronic Systems Design (MSD) program as their Block II in-house project. Since then, each MSD cohort has contributed to AutoRef's continuation in their respective Block II in-house projects. In TU/e's 2019–2020 academic year, a team of students in the Honors program High Tech Systems realized the control of drones as part of AutoRef's implementation, with continued efforts in drone control by these students leading to the Firefly-project.

Team contributions in AutoRef's development history are divided by a paradigm shift starting with the 2020–2022 MSD cohort project team (AutoRef PDEng MSD 2020 or simply MSD 2020) contribution in 2021. Unlike the teams which came before it, MSD 2020 eliminated the requirement of quadcopter drones as the basis for the autonomous refereeing system and emphasized continuity as a key stakeholder concern in system architecture. As such, the history of AutoRef is presented across:

  1. the 2015–2020 system based on various quadcopter drones; and
  2. the 2021–present system.

2016–2020 (drone-based system)

See also: Team contributions

From 2016 through 2020 AutoRef's development was based on camera-equipped quadcopter drones in autonomously refereeing RoboCup Middle Size League (MSL) robot soccer matches. Most of the 2016–2020 projects introduced their own systems engineering and implementation approaches. This repetitive work combined with the relatively short two-month duration available to these teams prevented the entire V-model from being realized for AutoRef and therefore causing relatively poor overall project progress across different teams.

Repetitions of the project definition (i.e., the left-hand side of the V-model) are specifically evident across the 2016–2020 team documentation. Earlier MSD cohorts emphasized a fully autonomous referee, while later years scoped the project towards an assistive, supplementary system for human referees. All teams from 2016–2020 specify the requirement that the AutoRef system be based on quadcopter drones. The majority of teams each used a different type of drone model in their implementation. This discontinuity in drone hardware between teams (among other hardware differences) further contributed to the slow progress AutoRef's development. To summarize the AutoRef 2016–2020 team contributions in terms of the underlying drone hardware:

  • MSD 2015 n/a (specified a drone-based system, but did not implement a specific type of drone)
  • MSD 2016 Parrot AR.Drone 2.0 Elite Edition
  • MSD 2017 custom-built drone based on a Pixhawk PX4 controller
  • MSD 2018 Avular Curiousity
  • MSD 2019 Crazyflie 2.X
  • Honors 2019 Crazyflie 2.1
Illustration by Peter van Dooren, BSc student at Mechanical Engineering, TU Eindhoven, November 2016.

2021–present

See also: Team contributions

In 2021, to prevent further repetitive efforts and slowed progress in the development of AutoRef, the MSD 2020 team was tasked with ensuring continuity in their contribution to the project. AutoRef's technical specification resumed that of an autonomous refereeing system as to fully replace human refereeing in MSL. The drone-based specification for AutoRef featured in previous team systems — that is, the requirement that AutoRef use drones — was also eliminated by MSD 2020.

As of MSD 2020, AutoRef's wiki pages (including the main, system architecture, and implementation pages) describe the current state of the project as a collective of team contributions. Whereas pre-MSD 2020 team contribution pages provide standalone descriptions of their respective architecture and implemention, pages for MSD 2020 onwards only summarize their contributions, keeping the detailed documentation on AutoRef's pages. An archive was also initiated by MSD 2020 to satisfy their stakeholder requirement for an overview of all team contributions, which is downloadable with access to AutoRef's TU/e SharePoint folder.

System architecture

Main article: AutoRef system architecture

Implementation

Main article: AutoRef implementation

Team contributions

Team contribution pages from 2016–2020 provide standalone descriptions of their respective architecture and implemention for AutoRef. Team contribution pages from 2021 onwards only summarize the team's respective contributions, keeping the detailed documentation on AutoRef's pages to ensure continuity. An archive provides a spreadsheet-based overview of all team contributions downloadable with access to AutoRef's TU/e SharePoint folder.

Teams 2016–2020 (drone-based system)

PDEng

High Tech Systems (HTS)

Teams 2021–present

PDEng

Archive overview of team contributions

Downloadable with access to AutoRef's TU/e SharePoint folder.

Retrieved from "https://dsdwiki.wtb.tue.nl/index.php?title=AutoRef_-_Autonomous_Referee_System&oldid=112893"