PRE2022 3 Group12: Difference between revisions
No edit summary |
No edit summary |
||
| Line 19: | Line 19: | ||
'''Policy Based Inference in Trick-Taking Card Games''' | '''Policy-Based Inference in Trick-Taking Card Games''' | ||
Summary: | Summary: | ||
| Line 27: | Line 27: | ||
Deterministic search algorithms such as Perfect Information Monte Carlo and Information Set Monte Carlo Tree Search have been employed. However, due to non-locality, deterministic search has been heavily criticized. To deal with these issues, inference helps by biasing state samples so that they are more realistic with respect to the opponent’s actions. Therefore, inference is a central concept in imperfect information games and plays a key role in the performance of deterministic search algorithms. This involves an opponent model to determine unknown information based on the action sequence. | Deterministic search algorithms such as Perfect Information Monte Carlo and Information Set Monte Carlo Tree Search have been employed. However, due to non-locality, deterministic search has been heavily criticized. To deal with these issues, inference helps by biasing state samples so that they are more realistic with respect to the opponent’s actions. Therefore, inference is a central concept in imperfect information games and plays a key role in the performance of deterministic search algorithms. This involves an opponent model to determine unknown information based on the action sequence. | ||
The paper describes how policy based inference was employed | The paper describes how policy-based inference was employed for Skat, which is a 3-player trick-taking card game and is played using a 32-card deck where cards 2 through 6 from each suit are removed from the standard 52-card deck. The actions are determined by three factors each having a certain probability. Firstly, the world relates to chance nodes in dealing and can be directly computed. Secondly, our own actions with a probability of 1 since we choose actions leading to a given state with full knowledge of our strategy. Lastly, a given state within the information set due to other players’ actions can only be determined perfectly if we have access to the other players’ policies. However, there are two issues. Firstly, we do not have access to the other players’ policies or they are computationally too expensive to model. Secondly, the number of states in the information set can be quite large. For these reasons, the authors suggest sampling the worlds and normalizing the distribution over the subset of states. | ||
The article concludes that | The article concludes that policy-based Inference appears to provide much stronger inference than other methods such as Kermit Inference and Card Location Inference. Furthermore, the authors conclude that sampling card configurations are more effective than sampling states. Lastly, it is suggested to experiment with heuristics that allow the algorithm to find states that are highly unlikely and discard them to improve the performance. | ||
Reference: | Reference: | ||
Revision as of 15:25, 11 February 2023
Problem Statement:
Society is currently faced with an ageing population. By around 2040, it is expected that one-quarter of the population will be aged 65 years or older. Compared to today, the size of this group of people will have increased by about 1.2 million people by 2040, all while the number of people working (in the age group 20 to 64 years old) will stay roughly the same. [1] This means a large shortage of healthcare workers will arise, causing some elderly to not receive all care they might be expecting. One important aspect of this care that might easily be overlooked are ways to combat their loneliness. This is often prevalent among the elderly, especially those aged 75 years or older. [2] One possible way to battle loneliness is to provide activities. However, with the reduced availability of care, it will become harder for healthcare workers to provide these activities. In these circumstances, robots can be used to support the workers.
Users:
User Requirements:
Approach:
Milestones:
Deliverables:
Task Division:
Literature Study:
Card games and AI
Policy-Based Inference in Trick-Taking Card Games
Summary:
This paper describes how an opponent model is used for inference in trick-taking card games, like Contract Bridge, Skat, and Hearts. These card games introduce uncertainty by featuring a large amount of private information, which becomes known after a long sequence of actions. Therefore, the number of histories is exponentially large in the action sequence length and extremely large information sets get created.
Deterministic search algorithms such as Perfect Information Monte Carlo and Information Set Monte Carlo Tree Search have been employed. However, due to non-locality, deterministic search has been heavily criticized. To deal with these issues, inference helps by biasing state samples so that they are more realistic with respect to the opponent’s actions. Therefore, inference is a central concept in imperfect information games and plays a key role in the performance of deterministic search algorithms. This involves an opponent model to determine unknown information based on the action sequence.
The paper describes how policy-based inference was employed for Skat, which is a 3-player trick-taking card game and is played using a 32-card deck where cards 2 through 6 from each suit are removed from the standard 52-card deck. The actions are determined by three factors each having a certain probability. Firstly, the world relates to chance nodes in dealing and can be directly computed. Secondly, our own actions with a probability of 1 since we choose actions leading to a given state with full knowledge of our strategy. Lastly, a given state within the information set due to other players’ actions can only be determined perfectly if we have access to the other players’ policies. However, there are two issues. Firstly, we do not have access to the other players’ policies or they are computationally too expensive to model. Secondly, the number of states in the information set can be quite large. For these reasons, the authors suggest sampling the worlds and normalizing the distribution over the subset of states.
The article concludes that policy-based Inference appears to provide much stronger inference than other methods such as Kermit Inference and Card Location Inference. Furthermore, the authors conclude that sampling card configurations are more effective than sampling states. Lastly, it is suggested to experiment with heuristics that allow the algorithm to find states that are highly unlikely and discard them to improve the performance.
Reference:
D. Rebstock, C. Solinas, M. Buro and N. R. Sturtevant, "Policy Based Inference in Trick-Taking Card Games," 2019 IEEE Conference on Games (CoG), London, UK, 2019, pp. 1-8, doi: 10.1109/CIG.2019.8848029.
A Social Robot as a Card Game Player
Summary:
Reference: Correia, F., Alves-Oliveira, P., Ribeiro, T., Melo, F., & Paiva, A. (2021). A Social Robot as a Card Game Player. Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, 13(1), 23-29. https://doi.org/10.1609/aiide.v13i1.12936
Exploring the Entertainment Value of Playing Games with a Humanoid Robot
Summary:
Reference: Johnson, D.O., Cuijpers, R.H., Pollmann, K. et al. Exploring the Entertainment Value of Playing Games with a Humanoid Robot. Int J of Soc Robotics 8, 247–269 (2016). https://doi.org/10.1007/s12369-015-0331-x
Object recognition
Papers that Luke will summarize:
- extension://elhekieabhbkpmcefcoobjddigjcaadp/https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=d0646c9b7c8111e1c7ed5b34827324c322c52638
Members:
- Abel Brasse (1509128) - a.m.brasse@student.tue.nl
- Linda Geraets (1565834) - l.j.m.geraets@student.tue.nl
- Sander van der Leek (1564226) - s.j.m.v.d.leek@student.tue.nl
- Tom van Liempd (1544098) - t.g.c.v.liempd@student.tue.nl
- Luke van Dongen (1535242) - l.h.m.v.dongen@student.tue.nl
- Tom van Eemeren (1755595) - t.v.eemeren@student.tue.nl