PRE2022 3 Group12/Literature

Card games and AI

Policy-Based Inference in Trick-Taking Card Games

Summary:

This paper describes how an opponent model is used for inference in trick-taking card games, like Contract Bridge, Skat, and Hearts. These card games introduce uncertainty by featuring a large amount of private information, which becomes known after a long sequence of actions. Therefore, the number of histories is exponentially large in the action sequence length and extremely large information sets get created.

Deterministic search algorithms such as Perfect Information Monte Carlo and Information Set Monte Carlo Tree Search have been employed. However, due to non-locality, deterministic search has been heavily criticized. To deal with these issues,  inference helps by biasing state samples so that they are more realistic with respect to the opponent’s actions. Therefore, inference is a central concept in imperfect information games and plays a key role in the performance of deterministic search algorithms. This involves an opponent model to determine unknown information based on the action sequence.

The paper describes how policy-based inference was employed for Skat, which is a 3-player trick-taking card game and is played using a 32-card deck where cards 2 through 6 from each suit are removed from the standard 52-card deck. The actions are determined by three factors each having a certain probability. Firstly,  the world relates to chance nodes in dealing and can be directly computed. Secondly, our own actions with a probability of 1 since we choose actions leading to a given state with full knowledge of our strategy. Lastly, a given state within the information set due to other players’ actions can only be determined perfectly if we have access to the other players’ policies. However, there are two issues. Firstly, we do not have access to the other players’ policies or they are computationally too expensive to model. Secondly, the number of states in the information set can be quite large. For these reasons, the authors suggest sampling the worlds and normalizing the distribution over the subset of states.

The article concludes that policy-based Inference appears to provide much stronger inference than other methods such as Kermit Inference and Card Location Inference. Furthermore, the authors conclude that sampling card configurations are more effective than sampling states. Lastly, it is suggested to experiment with heuristics that allow the algorithm to find states that are highly unlikely and discard them to improve the performance.

Reference:

D. Rebstock, C. Solinas, M. Buro and N. R. Sturtevant, "Policy Based Inference in Trick-Taking Card Games," 2019 IEEE Conference on Games (CoG), London, UK, 2019, pp. 1-8, doi: 10.1109/CIG.2019.8848029.

A Social Robot as a Card Game Player

Summary:

In this paper, it is investigated how a social robot player that is able to play the card game with social behaviours towards its partner and its opponents can be built. The authors state that generally, social robots can contribute with new ways of creating socially engaging interactions with humans in entertainment contexts.  For instance, physical embodiment can provide a more immersive user experience, an improved game feedback and a more believable social interaction. However, for multi-player games played in the physical world, the social environment becomes even more relevant. Therefore, the paper finds an answer to the question of how people will perceive a social robot player compared to human standards and if people are willing to trust a social robot to be their partner in a team game.

For their research, the authors employ a social robot, which is able to express emotions, provide spoken feedback, and respond socially, for the team card game called Sueca, which is a non-deterministic game and is considered an imperfect information game. Furthermore, this paper explores how the algorithm’s parametrizations affect the performance-time configuration.

The robot has two modules, the game module responsible for choosing moves and the social module responsible for social behaviour. The game module evolved over three stages. Firstly, a rule-based player was created that replicates the general gameplay strategy of non-professional human players. The performance of this agent was found to be similar to the performance of human players. Secondly, the Perfect Information Monte-Carlo (PIMC) algorithm was applied, which is a suitable algorithm for partially observable environments. With PMIC, the hidden information is sampled several times and the best move is computed by solving exactly or heuristically in each perfect information game by use of the MinMax algorithm. However, PMIC has two disadvantages, namely strategy fusion and non-locality. On top of that, success of PMIC depends on three game properties, namely leaf correlation, bias and disambiguation factor. Thirdly, a hybrid approach was employed to respect the time constraint without bounding the depth of the search. Here PIMC was only used from a certain tick on. Up to that tick, a stochastic version of the rule-based strategy was adopted.

Additionally, the social module ensures that the robotic player engages in social interactions using both verbal and non-verbal behaviours. The emotional and social behaviours of the robot were built by An emotional agent framework (FAtiMA) allowing the robot to not only play competitively but also respond emotionally in a natural manner by emotional appraisal, social and emotional behaviours.

The experiment was performed by using the autonomous robot EMotive headY System (EMYS) over a multi-touch table using physical cards. Furthermore, the approach was tested by a user study to compare the levels of trust that participants attributed to the robot. The results showed that human players increased their trust in the robot as their game partners. Furthermore, results have shown that the robot team had a winning rate of 60%.

The authors conclude that the social robot showed similar results in terms of trust from a human partner as humans partners,  indicating that we trust a robot to be our game partner in a card game. This also shows the success of the social human-robot interaction.

Reference:

Correia, F., Alves-Oliveira, P., Ribeiro, T., Melo, F., & Paiva, A. (2021). A Social Robot as a Card Game Player. Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, 13(1), 23-29. https://doi.org/10.1609/aiide.v13i1.12936

Exploring the Entertainment Value of Playing Games with a Humanoid Robot

Summary:

In this paper, a social robot plays the Mastermind game with a human to study how behavioural patterns of the robot affect the entertainment experience of a human player, especially when the behavioral patterns are tied to events in the game in a meaningful way. A major societal challenge is the ageing society due to an increased life expectancy as this decreases the support ratio resulting in a high pressure on caregivers. The authors mention three reasons why a social robot could provide social support in the form of entertainment. Firstly, studies have shown that this improves elderly people’s well-being significantly. Furthermore, an entertaining robot could engage its users in other tasks such as health exercises, taking their medicine on a regular basis or measuring health functions like blood pressure. Research has also shown that physical embodiment is more entertaining than playing against a screen-based character since it offers more diverse, richer forms of interaction.

The results showed that the robot’s behavior did not seem to affect the participants’ enjoyment of the game. However, the authors conclude that people experience playing games with a robot as entertaining. Furthermore, by combining speech, gestures, and eye LED patterns, robots can imitate very subtle levels of emotions that are correctly perceived by humans.

Reference:

Johnson, D.O., Cuijpers, R.H., Pollmann, K. et al. Exploring the Entertainment Value of Playing Games with a Humanoid Robot. Int J of Soc Robotics 8, 247–269 (2016). https://doi.org/10.1007/s12369-015-0331-x

Understanding the Success of Perfect Information Monte Carlo Sampling in Game Tree Search

Summary

In this paper, it is shown how it can be decided from game properties whether Perfect Information Monte Carlo (PIMC) for playing imperect information games that are too large to be optimally solved will be an effective approach to new and unexplored games. PIMC makes two types of errors, namely through the concepts of strategy fusion and non-locality. On top of that, it depends on the properties leaf correlation, bias and disambiguation factor of the game.

Reference

Long, J.R., Sturtevant, N.R., Buro, M., & Furtak, T. (2010). Understanding the Success of Perfect Information Monte Carlo Sampling in Game Tree Search. Proceedings of the AAAI Conference on Artificial Intelligence.

Object recognition

Real-Time Detection and Recognition of Cards in the Game of Set

In this paper a procedure is described for detecting and recognizing which cards of the game Set is visible in a real-time video feed. Object detection methods are used to detect the presence of cards in a video sequence and to recognize which cards of the game of Set are present by using the different features of the cards. The procedure consists of three steps. Firstly, the individual cards get extracted/segmented from the images of the real-time video feed. Secondly, a deep two-layered fully connected convolutional network performing feature extraction is used to separate each image. Finally, valid set combinations are found using feature interference in order to assign an image of a card to the correct set.

In this paper, it was the aim to recognize which cards were present as fast as possible. Hence, the card detection, feature extraction, and classification all needed to be performed fast. This had its consequences. In order to have fast feature extraction only a limited number of pixels of each image were used, making the feature extraction procedure less robust.

In this paper, cards of the game of Set were used. These cards have four different features: shading (solid, striped, or outline), shape (oval, squiggle, or diamond), color (red, purple, or green) and number of shapes (one, two, or three). For every card feature one classifier was trained. In addition, another classifier was trained to determine whether the segmented part of the image was a card of the game of Set at all. So, in order to train the classifiers, the representative dataset also consisted of some cards which were not part of the game of Set.

In the procedure of extracting the cards from the images of the real-time video, the OpenCV framework is used. OpenCV uses contour finding on grayscale images which separates the cards, which should be white, from the background, which should be black, by using grayscale images.

Reference:

Ditrih, H., Grgić, S., & Turković, L. (2021, September). Real-Time Detection and Recognition of Cards in the Game of Set. In 2021 International Symposium ELMAR (pp. 161-164). IEEE.

Pothole Detection using CNN and YOLO v7 Algorithm

In this paper, the YOLO v7 algorithm together with the Google API and an accelerometer is implemented in an app in order to identify and locate potholes. The YOLO v7 algorithm is used in order to categorize the potholes and the Google API and Accelerometer are used for counting the potholes. At the time, the YOLO v7 algorithm was the most recent version of the algorithm. The newest version, YOLO v8, has been released recently. However, not much work has been released using this version yet. Hence, this paper has been chosen.

The YOLO, which stands for ”You Just Look Once”, paradigm states that you only need to look once in order to recognize an object. YOLO v7 is a real-time object detector which has started a revolution in the way computer vision is practiced. With YOLO you can make accurate predictions about locations, classes and bounding boxes using visual frames, for  instance the frames of a (real-time) video. According to this paper, the YOLO algorithm is considered to be one of the most efficient methods for locating potholes, as it is both accurate and quick.

Considering the statements made about the effectiveness of the YOLO v7 algorithm for object detection in this paper, using this algorithm in our project might be a good idea.

Reference:

Reddy, E. S. T. K., & Rajaram, V. (2022, December). Pothole Detection using CNN and YOLO v7 Algorithm. In 2022 6th International Conference on Electronics, Communication and Aerospace Technology (pp. 1255-1260). IEEE.

Implementing Playing Cards BlackJack Game using OpenCV

In this paper, OpenCV is used to create software which detects and recognizes playing cards and decides which step is the best next step to perform in a game of Blackjack.

The OpenCV library is an open source computer vision and machine learning software library built to give a structure for computer vision applications. This library can be used for  recognizing faces, identifying object and much more. OpenCV is compatible with mostly every device and is provides many API’s.

In the literature study of this paper, a group of developers is discussed who used computer vision to identify playing cards during a poker game. The developers described a series of steps in their procedure of identifying playing cards Firstly, they extracted the card from the playing field by the contrast between the playing cards and the field they are resting on. Next, they used “template matching” to identify which cards are on the playing field. The program used by the developers uses probabilities to identify the card that it sees. This is possible due to the fact that a deck of playing cards has a finite number of possibilities, as there are only 52 cards in a deck.

In this paper, some challenges that needed to be faced in order to recognize cards successfully have been described. The lighting conditions should be good, because the definition of objects is influenced by lighting and bad lighting might influence the categorization of objects negatively. The background should be contrasting the objects as much as possible in order to prevent objects blending into the backdrop, making identification difficult. Objects might be only visible from a specific angle/viewpoint, this can make an object appear differently and hence influence the categorization of the object negatively. These challenges should be taken into account when performing object recognition.

Reference:

Mohd, T. K., Akkar, A., Cassens, J., Cregan, S., & Vander-Pallen, M. A. (2023). Implementing Playing Cards BlackJack Game using OpenCV.

Playing Card Recognition Using Rotational Invariant Template Matching

In this paper, a method for recognizing playing cards is described. In this paper, old playing card recognition methods have been extended such that cards that are rotated and/or have a different distance towards the camera also can be recognized successfully. By considering rotation and scaling robust playing card recognition can be obtained.

In this paper, a web camera is used in detecting the playing card. The camera acts as a sensor to scan the playing cards. It is found that the resolution of the camera is important. The higher resolution camera is used, the easier it is to achieve good results.

The playing cards are differentiated from the background by thresholding grey image. In addition, edge detection is used not only to find the edges of the cards, but also in order to find the contours of rank and suit.

A card can be recognized while being in any position in the background, only when rotation and scaling are considered in the process of card recognition. Hence, when doing the comparison between the obtained image and the card templates, the cards get rotated to the same position as the templates of the cards.

In the method used in the paper, character segmentation differentiates between card rank and card suit segmentation. Template matching is used on the segments and based on the degree of match it is decided which card is most likely in front of the camera.

In the paper, it is concluded that the same cards that are used to generate the templates need to be used when using the system, because the system cannot be successfully generalized to a wider variety of playing card decks when it is not trained on that.

Reference:

Zheng, C., & Green, R. (2007, December). Playing card recognition using rotational invariant template matching. In Proc. of Image and Vision Computing New Zealand (pp. 276-281).

Introducing Computers to Blackjack: Implementation of a Card Recognition System using Computer Vision Techniques

In this paper, a project is described in which a computer program is written which is capable of recognizing all cards of a standard deck of playing cards.

The playing card recognition procedure consists of multiple different components. Image thresholding was performed to differentiate between the cards and the black background by using a simple intensity histogram in RGB space. Grassfire transform was used to close up black regions in the cards. The grassfire transform was chosen because it is quick and simple. Then, a two-pass segmentation was used on the binary image to  segment out the different card regions and to find out how many cards were present in the image. Subsequently, card rotation was used to rotate the cards. In order to be able to perform card rotation data about the central moment of each card needs to be obtained and used. After that, a card thresholding process was used to process each card individually and the color of the card was recognized by producing an average intensity. Finally, after successfully producing the region maps of each card, both the suit and the number of each card was determined. Three simple networks were trained to distinguish between the three face cards.

In the paper, it is concluded that noise in the camera was the source of many misclassifications. Hence, the developers of the computer program implemented a voting scheme which runs the program for five frames before making a decision for each card. This design decision was made trying to eliminate the effect of the camera noise.

Reference:

Hollinger, G., Ward, N., & Everbach, E. C. (2003). Introducing computers to blackjack: Implementation of a card recognition system using computer vision techniques. Colby College, Waterville.

Poker card recognition with computer vision methods

The paper describes a method to recognize playing cards that does not make use of machine learning. Instead the paper describes the steps that can be taken to process an image such that the type of card can be determined based on mathematical properties.

An image of a set of cards that someone is holding, which is taken by a regular mobile phone can be processed using the following steps:

1. The RGB color-image is transformed into a grayscale image to reduce the amount of data, but retain the shapes that will be used later on for the detection.
2. Images often contain random noise, that might either be caused by the phone taking it or by the environment. To denoise the image, Gaussian filtering is used.
3. The grayscale image, with pixel values between 0 and 255, is binarized, meaning that each value is either black or white. This is done using a simple threshold that is picked based on the image.
4. The moment of the binary image is analyzed to get the region of the image containing the playing card. Moments are ways that describe the (geometric) features of an image.
5. Using Hu moments, finally the shape and symbols on the playing cards are recognized.

By applying above steps to known images, templates can be created that can be used later to detect actual playing cards.

Reference:

Hu, X., Yu, T., Wan, K., & Yuan, J. (2021, August). Poker card recognition with computer vision methods. In 2021 IEEE International Conference on Electronic Technology, Communication and Information (ICETCI) (pp. 11-15). IEEE. https://ieeexplore.ieee.org/document/9563607

Poker Watcher: Playing Card Detection Based on EfficientDet and Sandglass Block

The paper describes a way to detect playing cards from a camera feed using deep learning methods. In particular it is noted that for the available data, the cards can be quite small, which is why special care must be taken to achieve proper recognition.

The paper uses a neural network called EfficientDet, which is a single-shot neural network. There are also two-shot object detection algorithms, but these are less time-efficient and hence less suitable for real-time applications. The algorithm was specifically tweaked by changing certain blocks which it consists of by Sandglass blocks. These blocks are better able to preserve the high-dimensional representations of the images that are being recognized.

Lastly, the paper discusses methods of preprocessing. Because it is unknown from what angle the camera will be recording, steps are taken to transform the image such that it always creates a top-down view. This is achieved using a perspective projection.

Reference:

Chen, Q., Rigall, E., Wang, X., Fan, H., & Dong, J. (2020, December). Poker Watcher: Playing Card Detection Based on EfficientDet and Sandglass Block. In 2020 11th International Conference on Awareness Science and Technology (iCAST) (pp. 1-6). IEEE. https://ieeexplore.ieee.org/document/9319468

Object Detection in 20 Years: A Survey

The paper describes a few milestones made in the past 20 years for object detection:

1. Between 1990 and 2010, most object detection algorithms were built based on handcrafted features. Basic detectors used techniques like sliding windows and feature selection.
2. From 2012, object detection took a leap forward by using Convolutional Neural Networks (CNNs), which can be split up into two groups:
   1. Two-stage detectors make use of a “course-to-fine” process.
   2. Single-stage detectors operate using a single complete step.

A lot of different neural networks have been developed in the past decade, each one striving for a better accuracy and speed. Two-stage detectors over time have proven to give a higher accuracy, but are significantly slower than single-stage detectors.

A lot of advancements have been made over the years. A particular advancement that is worthy to note is that of lightweight network design. Some neural networks have been developed, taking in mind that they can be ran on lightweight edge devices: devices that might be placed inside a self-driving car or smart camera. These devices do not have a lot of computing power and hence are limited in the size of the neural network they can efficiently run. Neural networks can take this into account, by reducing the number of channels or parameters that each layer has to deal with, which makes it run faster, at the loss of accuracy.

Reference:

Zou, Z., Chen, K., Shi, Z., Guo, Y., & Ye, J. (2023). Object detection in 20 years: A survey. Proceedings of the IEEE. https://ieeexplore.ieee.org/document/10028728

A survey of modern deep learning based object detection models

This paper provides an overview of the popular neural networks that have been developed in the past decade. Some datasets are described which are commonly used to measure the performance of the neural networks. These datasets provide a large number of pre-labeled images which can be analyzed by the neural network to retrieve an accuracy score.

Similarly to the paper ‘Object Detection in 20 Years: A Survey’, some early non-neural-network-based object detection is described, as well as a number of two-stage detectors. Lastly, the single stage detector algorithms are described, which are most applicable to this project:

• YOLO is a single stage detector inspired from the GoogLeNet model for image classification. It surpassed other single stage models that existed at the time in both accuracy and speed. However, it had significant shortcomings in terms of localization accuracy for small or clustered objects. It also was limited in detecting multiple objects close together.
• SSD (Single Shot MultiBox Detector) was the first single stage detector to match the accuracy of other two-stage networks, which were known for better accuracy. It was also faster and more accurate than YOLO, but had similar problems detecting small objects.
• YOLOv2 was an improved version of YOLO, replacing large parts of the internal neural network structure. It had fewer internal parameters and hence offered better detection speed.
• RetinaNet is simple to train, converges faster and easy to implement. It achieved better performance in accuracy and run time than the two stage detectors. RetinaNet also pushed the envelope in advancing the ways object detectors are optimized by the introduction of a new loss function.
• YOLOv3 was another improved version of its predecessors, although no ground breaking changes were made.
• CenterNet brought a fresh perspective by not detecting objects using bounding boxes like other algorithms, but instead detecting them as single points. It is more accurate than the previously mentioned algorithms.
• EfficientDet (based on the EfficientNet backbone) is a scalable detector, which can scale dynamically based on the number of features. It again proved to have better efficiency and accuracy compared to it’s predecessors.
• YOLOv4 incorporated a lot of exciting ideas to design a fast and easy to train object detector that could work in existing production systems. It utilized a number of methods that increased the training time, but did not affect the inference time.
• YOLOv5 was another iteration of the YOLO algorithm, published by a company called Ultralytics. It was only released as a GitHub repository, and not as a peer-reviewed research, which caused doubts about the authenticity and effectiveness. However since it’s release, it has been utilized in several applications with effective results and has started to generate credibility.

Reference:

Zaidi, S. S. A., Ansari, M. S., Aslam, A., Kanwal, N., Asghar, M., & Lee, B. (2022). A survey of modern deep learning based object detection models. Digital Signal Processing, 103514. https://doi.org/10.1016/j.dsp.2022.103514

A Review of YOLO Algorithm Developments

The paper describes the many iterations the YOLO algorithm has gone through in the past years. It describes that the YOLO and YOLOv2 were not effective at detecting small targets. Among with other improvements, these issues were largely addressed with the release of YOLOv3. The newest version of YOLO (at the time the paper was written) was YOLOv5. This version of the neural network was much more flexible, and could easily be trained on your own dataset.

Reference:

Jiang, P., Ergu, D., Liu, F., Cai, Y., & Ma, B. (2022). A Review of Yolo algorithm developments. Procedia Computer Science, 199, 1066-1073. https://doi.org/10.1016/j.procs.2022.01.135

Problem, users and existing products

Designing social games for children and older adults: Two related case studies

Summary:

This article focuses on making a new board game, more engaging by adding technology. While focusing on two groups, namely children and elderly. The reasoning for combining the paper games and technology is, is that paper games have two important elements which games on computers either don’t have, or have less. So would the combination be able to still have the tangible parts, such as moving an actual piece across the board. As well as that the combination has more social interaction between players, as they can actually look each other in the face.

They based the gameboard versions for the children and elderly on what they usually play and their own suggestions. For the elderly they noticed that bank, poker and Black Jack were popular card games.

With the technology the playboard becomes more dynamic, sounds are added to create suspense for example, and the technology adds a bit of uncertainty to game. Both groups appreciated the uncertainty element, and found the version with the technology to be more engaging than the same paper version.

Reference:

A. A. Mahmud, O. Mubin, S. Shahid, J. Martens (2010). “Designing social games for children and older adults: Two related case studies”. https://doi.org/10.1016/j.entcom.2010.09.001

Friends or Foes? Socioemotional Support and Gaze Behaviors in Mixed Groups of Humans and Robots

Summary:

The study tries to figure out how humans and robots behave when interacting in small groups, with both humans and robots. And whether this could give new insights to how social robots should be designed. One robot that is used in the experiments is relationship driven and cooperative, namely Glin+. Whereas the other robot, Emys-, is competitive. For the experiment they focused on two elements, the eye gaze and the socioemotional support.

They concluded that participants looked more often at Glin+ when it was their partner, while they looked more at Emys- when it was their opponent. This could possibly be, because they consider Emys- to be more of a “threat” for their goal in the game, as Emys- is more competitive.

They also noticed that Emys- got more support when Emys- was a opponent, than Glin+ got when Glin+ was an opponent. This came as a surprise towards the authors, as they had expected that Emys- competitive behavior would have lead towards rivalry and would have undermined pro-social motivation. They suspect this might be, because the participants possibly wanted to appease Emys-.

The authors’ last conclusion is that partners supported each other more, and that the participants showed more support towards other humans.

Reference:

R. Oliveira, P. Arriaga, P. Alves-Oliveira, F. Correia, S. Petisca, A. Paiva (2018). “Friends or Foes? Socioemotional Support and Gaze Behaviors in Mixed Groups of Humans and Robots”. https://ieeexplore.ieee.org/document/9473499

Magic iCub: A Humanoid Robot Autonomously Catching Your Lies in a Card Game

Summary:

The article tries to create a robot that is autonomous, and is not driven with a Wizard of Oz approach. They try to make it autonomous by making the robots decisions based on the robot trying to measure a human’s inner state, they do this with eye tracking. Another goals of the robot is to do an entertaining activity with a human. For the game, the robot tries to guess which card is the secret card out of the 6 cards the human is describing. The human will pick 6 random cards, and pick 1 random card of the 6 to lie about when describing the cards. The card the human is lying about is the ‘secret card’. The cards don’t have any QR-codes, to reassure the human players that the robot could not cheat.

One of the problems that the robot has, is that the robot is sensitive to light-level changes. Although this is mainly an issue for outdoors, as light changes should not happen indoors.

In their conclusion they state that the robot is successfully able to measure a human’s inner state, as the robot has a high accuracy for guessing the correct secret card.  And that the robot is able to autonomously guide a human-robot interaction, as the measures of fun confirm that the game is entertaining.

Reference:

D. Pasquali, J. Gonzalez-Billandon, F. Rea, G. Sandini, A. Alessandra Sciutti (2021) . “Magic iCub: A Humanoid Robot Autonomously Catching Your Lies in a Card Game”. https://doi.org/10.1145/3434073.3444682

Just follow the suit! Trust in Human-Robot Interactions during Card Game Playing

Summary:

The paper’s aim was to create a social robot and an entertaining activity to reconnect the elderly, who often suffer from social isolation. They also hoped that it would contribute to the QoL elderly experience.

For their project they chose one of the most played games among elderly in Portugal, namely Sueca. Since robots become more competent, humans might start seeing them as fierce competitors. But people might still be ware of robots. Therefore, the paper tries to figure out how trust levels could work between humans and robots.

The authors concluded that humans do trust robots. But the level of trust they have in the robot, depends on previous encounters with the robot. Therefore, suggesting that to increase the trust level there has to be a longer period of being acquainted.

Reference:  

F. Correia, P. Alves-Oliveira, N. Maia, T. Ribeiro, S. Petisca, F. S. Melo, A. Paiva (2016). “Just follow the suit! Trust in Human-Robot Interactions during Card Game Playing”. 10.1109/ROMAN.2016.7745165

Motivational Factors for Mobile Serious Games for Elderly Users

Summary:

This paper focuses on understanding what motivates elderly users to use (serious) games. Serious games refer to game whose goal is not just amusement, so for example games that have also goal teaching the player something new.  The primary reason, users use a game is the usability of it. So games have to be adjusted to the needs of the elderly, such that it is easy to use. Elderly can namely have trouble with sight, hearing, attention, motor skills and using technology. Other motivations to use a game are that it is fun, relaxing, social interaction, gives regularly motivational messages, a good balance between ability and difficulty, and personalization of levels and time. Elderly also play games as a way to escape reality.

Reference:

R. N. S. de Carvalho, L. Ishitani (2012) “Motivational Factors for Mobile Serious Games for Elderly Users”. https://www.researchgate.net/publication/277709317_Motivational_Factors_for_Mobile_Serious_Games_for_Elderly_Users

Designing and Evaluating the Tabletop Game Experience for Senior Citizens

gaming is widely experienced as a means for social interaction. Especially for elderly with less leisure time. Elderly mostly play low-tech games such as card games. They are less inclined towards computer based games, this might be because they are unfamiliar with the newer games and more afraid of the complexity of both installation and playing.

Another point might be that games do not aim at elderly people and thus make games that are unfamiliar and hard to understand.

According to their field-research at an local community center elderly mostly play card games (e.g., bank, poker), guessing games and memory games. The majority of the games seemed as having simple and uncomplicated rules. Most elderly also moved between groups of different people as they play different games, most of which revolved around the tables.

From interviews it seemed that their game play was mostly of social nature, and that they disliked playing for money and avoided gambling. Primary motivation to play games is to have fun and to widen their social network.

al Mahmud, A., Mubin, O., Shahid, S., & Martens, J.-B. (2008). Designing and Evaluating the Tabletop Game Experience for Senior Citizens. https://dl.acm.org/doi/pdf/10.1145/1463160.1463205

A Single-User Tabletop Card Game System for Older Persons: General Lessons Learned From an In-Situ Study

This paper is about a study on the use of tabletop systems for card playing aimed at senior citizen with little to no computer experience. Researched whether the use of an alternative form of Briscola (card game) could be offered to older people who were (temporarily) restricted to their homes, or for some reason were unable or uninclined to play with others. The study was done in a home where participants often used the system with friends resulting in less information about isolated use.

They found that their BriscolaTable was used even though they could choose freely between other activities at the center, total of 67 games from 22 players in 2 weeks.

improvements could be made by paying more attention to motor limitations associated with age, specifically the drag and drop on a screen and the deterioration of touch in older adults.

They assume the ease of use that their participants experienced was mostly due to the correspondence between actions of the Table and normal card-playing. But as in general literature (according to this article) the consensus that it is unnecessary to stick rigidly to those metaphors, they found that they could have done more e.g. to include more ‘shortcuts’ (deal cards button, instead of manually dealing all cards 1 by 1), while also mentioning that some users would prefer to do some physical actions by themselves.

The study also found that the elderly found the agent that generally only canned utterances to be entertaining (even though not actually very useful). Probably also due to their ‘spectating friends’.

Gabrielli, S., Bellutti, S., Jameson, A., Leonardi, C., & Zancanaro, M. (2008). A single-user tabletop card game system for older persons: General lessons learned from an in-situ study. 2008 IEEE International Workshop on Horizontal Interactive Human Computer System, TABLETOP 2008, 85–88. https://doi.org/10.1109/TABLETOP.2008.4660188

How older people account for their experiences with interactive technology

This paper aims to complement existing work with a discussion of how people who are just starting to use computing technology account for the difficulties they encounter. This because the problems are not confined to physical and cognitive factors; attitude, anxiety, perceived relevance of the technology to everyday life, usefulness, usability,….

This was done as study sessions where older people learned to work with a personal computer, where they discussed the problems they encountered.

Reports of bad experiences; unpredictability of individual features, to frustration at ones own inability to remember necessary sequences of operations. Most of the bad experiences seem to relate to the perceived uncontrollability. But some themes were found among them; alienation (‘This is not my world at all’), identity (‘I worked in a job with people, not with machines’), agency (‘But sometimes you’re obliged to’), anxiety (‘I was frightened to’), age related (‘being too old’), being too busy (‘You haven’t got a space in the day to learn’), finding a purpose for the technology (‘I see their uses but I don’t have to accept them fully’).

Turner, P., Turner, S., & van de Walle, G. (2007). How older people account for their experiences with interactive technology. Behaviour and Information Technology, 26(4), 287–296. https://doi.org/10.1080/01449290601173499

It’s Food Fight! Designing the Chef’s Hat Card Game for Affective-Aware HRI

This paper describes the design of an interactive game between humans and a robot that makes it possible to observe, analyze, and model competitive strategies and affective interactions with the aim to dynamically generate appropriate responses or initiations of a robot.

The requirements they set for The Chef’s hat game were;

1.    The game should elicit a multitude of affected behaviors that the robot can properly understand and model.

2.    The game should be playable without the need of verbal expressions.

3.    The game should provide the possibilities of strategies based on bonding between players.

4.    The game should have specific mechanics that cause affective reactions.

5.    The game should be easy to follow and understand, clear turns and small number of actions.

6.    The game status should be easy to track and monitor.

7.    The game should give players enough opportunities to interact with other players (through game-mechanics).

Their design used a top-view camera that is used to monitor the entire game state, that way the robots can make decisions by analyzing a single image. The playing cards they used were given QR-codes so the robot could easily recognize them.

The goal for the robot involved in the game will be to understand the affective behavior of the human participants and use this information, together with that of the game status, and to select the best strategy to play the game.

This study also mentions the quality of the robots strategy, having it make accurate predictions for some (5) steps into the future could make it win with high likelihood. This would not make for an enjoyable gaming experience, the same can be said when the robot tries to let the player win al the time. To combat this they use an adaptive system that implements a strategy based on the emotional state of the other players.

Barros, P., Sciutti, A., Bloem, A. C., Hootsmans, I. M., Opheij, L. M., Toebosch, R. H. A., Barakova, E., & Toebosch, R. H. (2021). It’s Food Fight! Designing the Chef’s Hat Card Game for Affective-Aware HRI. https://doi.org/10.1145/3434074.3447227

Robot Design

Design of a Card Picking Robot

The aim of this paper is to explain how they designed a robot arm which is able to pick up cards from a stack and move them under a camera-scanner. They explain 5 candidate designs and discuss their benefits and pitfalls.

In all of their designs they use a suction cup which is connected to an air pump with a solenoid valve to create and release a vacuum on demand.

1. The first design was a 'plotter' robot (robot arm on a stationary frame). This design was easier to build than other designs since they owned an existing plotter but suffered from slow speeds and imprecise movement.

1. The second design was an 'air cylinder' robot, this design featured a stationary robot arm which can rotate on its axis and move the suction cup up or down (2-DOF). The main feature of this design is that all movement was achieved using compressed air.
2. The design scored high on precision but the cost was prohibitively expensive.
3. The third design was the 'rack and pinion' robot, similarly to an overhead crane this design has a stationary frame hanging above which guides the arm on a belt. But this design was deemed too difficult to manufacture.
4. The fourth design was the 'photocopier' robot. Similarly to a printer device all of the cards would be held in a magazine under some roller wheels and would be moved downhill to the scanner. This design was reliable but too costly to manufacture. (Besides this design would not fit our card-playing specifications since we need 2-way movement.)
5. The fifth design was the 'stepper motor' robot, this design uses a central shaft which can rotate the entire arm using a precise stepper motor, the arm can move up and down using a lever attached to a solenoid. This design was precise and affordable using mostly commercially available parts. After comparing the scores they decided that this stepper motor design was deemed the overall best and they decide to use this in their final design.

Diagram of their final 'stepper motor' design.

They use an innovative system which they call The Vibrabox. This uses a steady air-flow over a stack of cards which makes the top card float a few millimetres above the other cards, which makes sure that exactly 1 card is picked up at a time when grabbed by the suction cup.

The paper further gives a detailed list of sub-components, diagrams of how the inside of the robot looked like, and their robot controller code.

We will also require a system to read and classify playing cards just like their analysis plate, so we will keep a close eye on this design.

Overall this last design is nearly a perfect match to our needs in this project. We will adapt this design further to specialise it for playing a card game.

Bunyan-Ngwiri, D. (1992). Design of a Card Picking Robot. [1]

Multi-DOF counterbalance mechanism for a service robot arm

This paper discusses how to produce a a low-cost robot arm for any desired DOF using springs and wires to counterbalance the movement and reduce torque.

Each joint is compactly fitted with the counterbalance mechanism and a separate actuator.

This allows the robot to be powered by low-power/cheaper actuators since they need to fight the robot's inertia less.

The caveat of this design is that the 'payload' of the arm is limited by the strength of the actuators, but since we plan on moving only lightweight cards this shouldn't be an issue.

This design could be viable to replace the 'stepper motor' arm of the above paper if we require a higher DOF robot.

Kim, H. S., & Song, J. B. (2014). Multi-DOF counterbalance mechanism for a service robot arm. IEEE/ASME Transactions on Mechatronics, 19(6), 1756-1763.

Design of a 3 DoF robotic arm

This short paper gives a mathematical analysis of the inverse kinematics of a 3-DOF robot arm.

This can be useful if we decide to use a 3-DOF arm in our design. It gives some useful equations on how to move the arm in 3d space when we have individual control over all 3 individual joints.

Ramish, S. B. Hussain and F. Kanwal, "Design of a 3 DoF robotic arm," 2016 Sixth International Conference on Innovative Computing Technology (INTECH), Dublin, Ireland, 2016, pp. 145-149, doi: 10.1109/INTECH.2016.7845007.

General purpose hands for bin-picking robots

This paper describes how an advanced grabber design can be produced for the task of picking up objects from an unorganised bin.

The 2 main grabber methods of the paper are the clamping and the attracting methods.

First the paper describes the most important design requirements.

Then they describe 3 designs which accomplish the requirements:

1. The surface-adapting vacuum gripper (SAVG) uses a suction cup on a semi-rigid spring which can use a vacuum to stick to a flat surface of an object. This design also features contact sensing and a locking mechanism.
2. The contour-adapting vacuum gripper (CAVG) worked similarly to the SAVG but uses 20 smaller suction cups in a rectangle shape to have more grip on the object.
3. The parallel-jaw hand was a traditional gripper design with 2 parallel 'fingers' and which can move towards each other to grab an object from the sides.

Overall of these design the first SAVG will be sufficient for our project.

The cards in our project will always be picked up in the horizontal position so the parallel-jaw hand would not work well and the CAVG would be overkill since we do not need much force to lift the light cards.

Tella, R., Birk, J. R., & Kelley, R. B. (1982). General purpose hands for bin-picking robots. IEEE Transactions on Systems, Man, and Cybernetics, 12(6), 828-837.

Picking up operation of thin objects by robot arm with two-fingered parallel soft gripper

This paper gives a detailed sequence of events and mathematical force analysis to pick up paper or a card using only soft grippers.

In this paper they used a multi-DOF robot arm with 2 soft 'fingers', one with a hard and pointy 'nail' which can be used to slide under the paper.

The success rate for picking up a paper sheet is 90% but unfortunately only 10% when picking up a hard plastic card.

Given the very low success rate on rigid cards which we will use and the relatively costly components, we will probably not use soft grippers in our project so in hindsight this paper will not be very useful.

Yoshimi, T., Iwata, N., Mizukawa, M., & Ando, Y. (2012, May). Picking up operation of thin objects by robot arm with two-fingered parallel soft gripper. In 2012 IEEE Workshop on Advanced Robotics and its Social Impacts (ARSO) (pp. 7-12). IEEE.