PRE2020 3 Group12

Robot Appearance

Group Members

Name	Student ID
Bart Bronsgeest	1370871
Mihail Tifrea	1317415
Marco Pleket	1295713
Robert Scholman	1317989
Jeroen Sies	0947953

Introduction

Probleem schetsen, eventueel mogelijke oplossingen aanstippen? Waarom het relevant is, waarom we er uberhaupt een oplossing voor nodig hebben Marco

For the longest time in human history, humans have seized every opportunity they could find to automate and make their lives easier. This already started in the classical period with a very famous example, where the Romans would build large bridge-like waterways, aqueducts, to automatically transport water from outer areas to Rome, automated by the force of gravity and the general water cycle of nature. Today, computational sciences opening ways for AI and robotics have opened many new opportunities for this automation. Think about all the robots that are used in production to do the same programmed task over and over again, classification of images (often better than human results) with deep learning networks, and more recently, the combined robotics and AI task of self-driving cars!

Slowly but surely, robots can take over repetitive, task-specified jobs, and do them more efficiently than their human counterparts. Currently, however, such jobs are very concrete. For robots to have more versatility, they simply need to become more like us. Robots should be adaptive, be able to learn from their mistakes, and handle anomalies efficiently; essentially, a robot should have a level of decision-making and freedom equivalent to that of a human being. For example, when a robot is specifically tasked with picking up a football from the ground, it should find out a way to get the football back when it's accidentally thrown in a tree. Another issue arises in this social situation as well: the robot has to have a certain level of flexibility in its movement, since it should not just be able to move around and pick up a ball, but also to use tools that help it in fulfilling its "out of the ordinary" task. All in all, one thing becomes very clear: These tasks can not be hardcoded, they have to be learned.

It stands to reason that building a robot with all implementations specified above, a robot suitable in a social environment, requires a humanoid design. This causes many challenges, among which the most notable would be the challenges grouped into Electrical/Mechanical Engineering and Computer/Data Science. A human has many joints and muscles for various movements, expressions, and goals. The human face in particular is an incredibly complex design, consisting of 43 muscles, using about 10 of those muscles to smile and about 30 to simply laugh. All these joints and muscles have to work together perfectly since every joint's state influences the entire structure of joints and muscles. One huge aspect in the collaborative work of these joints and muscles is our sense of balance, meaning how the robot prevents itself from falling over. At every iteration, the robot has to check the state of its balance and decide which joints and muscles need to be given a task for the next iteration. Additionally, there is the Computer Science-related challenge of mapping such an unpredictable environment to binary code interpretable by the robot. Every signal from its surroundings has to be carefully processed and divided accordingly. The robot has to learn, diving into AI, and more specifically, Reinforcement Learning. Reinforcement Learning currently only exists as a solution to very basic learning problems, where the rules are very concretely specified, and suffers from the environment being too broad in terms of data. This is why it is still a very experimental field, and many robot studies include abstract definitions of states and policies for environment data that heavily imply the usage of Reinforcement Learning in theory, but lack a practical implementation.

As a consequence, social/humanoid robots are still a widely researched topic. The research has for example huge usage potential in care facilities and other forms of social care for the elderly, people with a disability, or even in the battle against loneliness and depression. Many papers cover how the robot could be taught certain skills with respect to the environment (Reinforcement Learning) and how it would communicate with humans. However, an oftentimes overlooked question is whether this robot would be accepted by humans at all. Many studies have found that a robot in the shape of a cute animal -a more easily modeled robot than a humanoid robot because humans expect less from the robot and the mimicked behavior is often very repetitive and standard- has a positive impact on its environment. People like engaging with the robot, as some see it as a kind of pet, and it successfully requests the attention of its audience. But, as we have established, such a robot would be very limited in the tasks it could fulfill, and modeling a humanoid robot is not as easy as modeling the behavior of an animal because we expect a lot more detail from the humanoid robot being humanoid beings ourselves. What would this robot look like to make it appealing instead of scary?

This phenomenon, a robot becoming scary as it approaches human-likeness, is what is referred to as the Uncanny Valley. Humans are able to relate to objects that act and look in a certain human way (generally humanoid), but something strange happens when we approach reality. If a robot or screen-captured character is manually made to look exactly like a human, we are often scared, disgusted, and 'uncanny' of it. It's as if we hate that it tries to fool us for being human, when we know it isn't from the various signs we pick up on as red flags. Because modeling a perfect human is impossible with the current state-of-the-art, designers try to refrain from creating a very human-like robot even if it has to be humanoid to properly fulfill its tasks. However, in a social situation, a robot has to be appealing enough for us to appreciate, value, and even accept it in our environment. On the other hand, though, if the robot is too 'cute', we tend to not open up to the robot at all, but rather use our attention to watch it, actively 'awwh' it, and help it in any way we can; not very useful when the purpose is to help you, although this can be helpful in battling loneliness.

Since most of this technology is still very experimental, it is important that we already start to define the variables involved in determining this Uncanny Valley with the examples we can already offer. Such examples are already widely available, not only due to attempts in developing human robots, but because of the art of recreating a human face in CGI (Computer Generated Imagery). By defining variables for various attributes of the appearance of a robots and its results on humans, or even just elderly, adults and students, we can help designers fall into the same pitfalls that others already have. Designers can decide on the appearance of a robot based on what exactly they need the robot to be able to do: a center of appearance studies.

Relevance of robot appearance

In a social context, robots may be subject to judgement from humans based on their appearance Walters, M.L., Syrdal, D.S., Dautenhahn, K. et al. Avoiding the uncanny valley: robot appearance, personality and consistency of behavior in an attention-seeking home scenario for a robot companion. (2008). The perceived intelligence of the robot is correlated to the attractiveness of the robot since it is the case that humans make a ‘mental model’ of the robot during social interaction and adjust their expectations accordingly:

• “If the appearance and the behavior of the robot are more advanced than the true state of the robot, hen people will tend to judge the robot as dishonest as the (social) signals being emitted by the robot, and unconsciously assessed by humans, will be misleading. On the other hand, if the appearance and behavior of the robot are unconsciously signaling that the robot is less attentive, socially or physically capable than it actually is, then humans may misunderstand or not take advantage of the robot to its full abilities.”

It is thus very important to predict and attribute the correct level of attractiveness depending on the intellectual capabilities of a robot. Not only that, but humans attribute different levels of trust and satisfaction when dealing with robots, depending on how much they like it Li, D., Rau, P.L.P. & Li, Y. A Cross-cultural Study: Effect of Robot Appearance and Task. (2010). Furthermore, an anthropomorphic robot is said to be better when high sociability tasks are required Lohse M et al (2007) What can I do for you? Appearance and application of robots. In: Artificial intelligence and simulation of behaviour, which is a statement that does not lack controversy as some research did not find any conclusive evidence of this aspect Li, D., Rau, P.L.P. & Li, Y. A Cross-cultural Study: Effect of Robot Appearance and Task. (2010).

Study objective

As many have regarded the uncanny valley the pitfall of humanoid robots, there is the necessity of figuring out how the appearance should change in order to climb this valley. The answer to this research question is rather simple, namely: to become indistinguishable from humans. But the problem of this simple answer is that there is no way to quantify the similarity in appearance to the one of humans other than measuring the uncanniness of the robot using humans and surveys. Better ways need to be researched in order to predict the similarity of a robot to the one of a human, based on robot features. For this, we intend to come up with a scoring system that can accurately predict where the robot sits on the uncanny valley. We are interested in computing this score using latent variables inferred from our research.

Theoretical Exploration

State of the Art

Hand-and-Finger Movement

In 2018 the non-profit research company OpenAI created a realistic human-hand-shaped robotic hand with a level of dexterity similar to that of a human [1]. The system that is responsible for the movement of the robotic hand uses a reinforcement model, where the AI is trained via trial and error. The hand itself is able to grasp and manipulate objects with state-of-the-art precision. Moreover, the hand exhibits unprecedented levels of dexterity as multiple grasp types that are found in humans can also be found in its hand movements.

Realistic facial expressions and head movement: Geminoid H1, H2, F and DK

The geminoid DK is a robot with a very anthropomorphic design, created to examine how the presence, the appearance, the behavior and the personality traits of an anthropomorphic robot affects the communication with human partners. The robot mimics the external appearance and the facial characteristics of the original, being its creator and a Danish professor Hendrik Scharfe. Apart from the movements in the robot’s facial expressions and head it is not able to move on it’s own (own = remotely by the operator). The Geminoid DK does not possess any intelligence of itself and has to be remotely controlled by an operator. Pre-programmed sequences of movements can be executed for subtle motions such as blinking and breathing. Moreover, speech of the operator can be transmitted through the computer network of the geminoid to a speaker located inside the robot.

Human Perception of others

Human beings interact which each other on a very frequent basis. These interactions can be partially explained by using psychological, social psychological and behavioral theories that have been developed over the years. It is of utter importance to understand how exactly humans interact, because many robot developers aim to create robots that are as close to an actual human as possible. Understanding how humans interact with computers and/or technology is not enough, there is an important human factor that needs investigation.

Method

All participants will be asked to fill out the questionnaire. In this questionnaire, subjects will see an image of a robot, and a video of the same robot moving, for three different robots. These robots fall into three different categories: slightly humanlike, humanlike and extremely humanlike. These three categories will be illustrated by the following robots: Atlas (Boston Dynamics), Honda ASIMO (Honda), and Geminoid DK (Aalborg University, Osaka University, Kokoro & ATR) // Sophia (Hanson robotics). To ensure that confounding variables like habituation to the robot have a minimal effect on participant judgement, the order of presentation is counterbalanced. This means that half of all participants will first see the image, then the video (normal order), and the other half will first see the video, then the image (reversed order). The order in which the different robots are presented is randomized. For a visual representation, please refer to table 1.

Subject responses will be measured using the Robotic Social Attributes Scale (RoSAS), developed by Carpinella, M. et al. (2017). Each robot will be rated on its warmth, competence and discomfort. Each of these three factors contains six items on which the robot will be evaluated. See table 1 for more details.

Enterprise

Users, society and enterprise

users, society and enterprise: Een algemene uitleg over de user geven en concrete users verzinnen en de problemen beschrijven enzo. En dan zonder je technologie te gebruiken laten zien hoe die personen zou kunnen benefitten van onze solution. Marco

The following user groups are involved with the appearance of a robot and would benefit from a consultancy in robot appearance:

Researchers

Researchers will benefit from such a consultancy because researchers can more carefully specify their goals. Generally pushing the state-of-the-art and therefore approaching human-like robots in their research, having a clear view on what can make a robot fall into the Uncanny Valley can help researchers to focus on what prevents those robots to fall into the Uncanny Valley as well. If the goal is better defined, it is easier to reach it.

Clementine, 34 and researcher at TU/e, has made amazing progress with her research! She has successfully found a way for a robot to learn from its environment, much like humans do when they are young. She has created a successful reward and policy system, by which the robot rates all actions it can take and potential better alternatives. Additionally, the robot has implemented balance, and can effortlessly align all its joints for smooth movement. However, the robot is only able to do practical tasks and does not have any form of reading humans, nor the ability to match certain emotions in a social group. The robot would only be required hold basic conversations with humans and is mainly meant for its practical use, but it does need to be accepted and trusted in a social circle. While not complex, trying to have the robot look human in order to be accepted among humans opens the risk of falling in the Uncanny Valley, and therefore, a different approach has to be considered. But Clementine finds it hard to determine what she could implement for facial expressions and face design to have communication go smoothly, but also be relatable or even cute enough to be associated with from a human perspective. To finalize her research, however, she can't overthink it, and Clementine is forced to finish her initial design, hoping the Uncanny Valley is completely avoided.

Designers

Designers will benefit from the various variables that are defined in research, on which the consultancy will be based. Generally staying in front of the Uncanny Valley, they have more freedom in what their robot will look like and could, with our help, very easily select the variables that matter based on the tasks their robot will be designed to fulfill. Designers can design their robots for their intended clients, without risking finalizing a design unappealing to this audience.

Rory, 43 years old, is a lead designer in a team that develops a home buddy. Meant to fight loneliness, this robot does not require too many practical abilities, but it does need a lot of information on humans, their emotions, and human language. Rory has managed to use image classification along with already developed AI for speech recognition to recognize human emotions and, in a way, understand what the person is saying. However, for humans to relate with robots on an emotional level, robots need to have a specific kind of human-ness and cuteness, without frightening the owner. However, avoiding the Uncanny Valley is hard, and several prototypes will have to be made and judged by his team and others before the product can go on the market. After a tiring performance test, where it is specially tested how well received the robot is in social circles, he can finally decide which of the robots can go on the market. When the correct design has been picked, Rory wants to mass-produce it by building production engines that will automatically assemble the robot, but for now, he will have to design every single prototype separately.

Clients

Clients will not have to deal with uncanny robots designed to help them again. This will ensure that no clients will have to fear or feel uneasy with an experimental social robot, which is in their best interest due to the costs and manpower required to care for them that can now be replaced by robots. With a perfectly designed robot in terms of appearance, issues that can not be dealt with very well in the present-day, such as loneliness and depression, can now be battled using these care robots.

Henry, 87 years old, has become mobility impaired over the years. Ever since his wife died, he loved walking by his daughter and grandchildren, who live a few blocks away. But sadly, visiting them has become harder and harder, and he more often than not chooses to not go out, but rather stay at home. Recently, his grandchildren started studying in the city, and his daughter has made a recent career development that forces a lot of time from her. Henry is very proud of his children and grandchildren, but struggles more and more to take care of himself and his house. He also misses visiting his relatives often, and even though he still has old friends of his come over every once in a while, being alone with only the tv gets tiring very quickly. He has thought about getting a pet, but Henry is afraid that he can't take care of the pet as he would hope. Because he is still fit enough to stay in, the local care home could not help him much further, since the best they could do is sending a volunteer every once in a while to check up on him. Life doesn't get Henry down very fast, but even he is starting to realize that these last years have been getting more lonely. All Henry wishes for right now is a buddy who could stay with him for a while.

Social volunteers

Social volunteers will have a massive advantage for this consultancy as a consequence of the state-of-the-art being pushed forward. When care robots in certain more concrete situations can be developed sooner using the well-defined variables for appearance, they can take over 'smaller' tasks of a social volunteer. Social volunteers often like doing their work a lot, but with the shortage it can take a lot of time, which is something not all volunteers have to offer too much of. Using care robots, social volunteers can spend their time elsewhere, maybe even by engaging with the cared-for even more than before.

Implementation details

Implementation details: Uitleggen hoe het bedrijf te werk zou kunnen gaan. Later komt hier ook het resultaat van de research bij Jeroen

We want to create an online platform on which users can receive feedback on the competence and likeability of a robot. There is much data on this topic available at the moment, for example how the appearance of a robot affects humans reactions towards it. Though the combination of the appearance and actual movement is much harder to determine. On our platform we offer two types of services. Users can get assisted on the design of their robot. They will receive feedback during their design process with multiple interaction points with our data analysts. Users can also request a final analysis when the product is finished. The specific terms can then be negotiated, but it could be done in the form of them sending a finished design model that we could judge or one of our analysts could travel to the actual robot to see the performance in real life.

The most amount of work will be the setup of the whole platform and the gathering of the data. For the former we need an intuitive interface. It should be easy for customers to submit and they should be able to share information that might help with the consultancy beforehand. We will support video content to be uploaded along with some easy to answer questions like the purpose of the robot and the target audience. Then the first appointment can be planned, which will be in the form of an online call. During this meeting the initial needs of the client will be assesed and an initial feedback will be given on the design. Based on the users' needs the next actions will be set up to guide them as well as possible.

The data needed to be able to perform such an assessment will be gathered beforehand. A user research will be conducted among people of different age groups. In this research we aim to be able to map the likeability of robots of various levels of humanness compared to their way of moving. At first we do not focus on a specific age group yet. We will transform the data gathered into a usable model that we can use to assess robots in general. Later we plan on expanding the research to increase accuracy. We would like to add the age group of the end user in our model as well as the role the robot will be fulfill, as we expect both to have a significant influence on how much a robot will be liked.

After this initial setup there are a couple key factors to the whole process. First of all we need to maintain the platform and employ enough consultants to serve our customers. Second of all we need publicity, the service we provide is very specific, so it is of much importance that we are able to reach our target audience. At last we would like to improve the current model. As said above, we would like to expant it to other age groups and specify it towards different functionalities for the robot. For this we need more research and more data.

Enterprise structure

Enterprise structure: hierarchy, what kind of company do we need to have? Robert

The enterprise itself will consist of just a few people and will mostly be located online. Clients will be able to send in their robot designs and they would be rated and evaluated online. After the robot attractiveness scale has been created it will not quickly age and therefore we do a need for need full-time data analysts. We do need one or more employees to perform maintenance to the website/platform we will be using and one or more employees that are taking care of advertisements. Thus the body of employees will be relatively small.

Cost and profit

Cost and profit: Evaluate the costs and profit and explain/discuss why it is a decent model for earning revenue. Robert

Since the enterprise will be mostly located online we can decide against hiring or buying an office. There are a benefits to having an office, for example you can have a central meeting point, and therefore we only decide against getting an office if the profits won’t allow for it. If we do so, we might need to hire computers, which can be located anywhere, for hosting the websites and running the evaluations. We should be able to hire only specific amounts of processing power of a computer, therefore we could reduce costs. The evaluation service we offer shall not be expensive, this is because we assume most anthropomorphic robot manufacturers already have a team designated to the design of the robot. Thus we cannot ask too much money, since then they will just solely rely on the designated team for design evaluation. However, making little profit is not necessarily bad, since the costs for maintaining the enterprise and performing the evaluation are also very low.

Aspect of the platform used by clients

Aspect of the platform used by clients: waarom het platform goed werkt voor onze target users. Jeroen

Funding campaign

Funding for starting the enterprise: how should we get it and for what should it be used? Robert

To successfully start up the enterprise we would need to get some funding from other organizations. Organization that might want to fund the enterprise are stakeholders that would benefit from the service the enterprise offers, like a robotics manufacturer or a care home that uses robots. This funding would be mostly used to hire data analysts to develop the robot attractiveness scales and for the advertisement of the company. These are necessary for the enterprise until it is able to manage and survive on its own. A campaign plan would show what steps would benefit and help start up the enterprise, therefore we have laid out such a campaign plan below.

Step 1: What are the enterprises concise and unique selling points?

Step 2: Approach to market. How do you intend to go about selling?

Since our service (evaluation of robot attractiveness) is quite niche we have to specifically target large companies. Any large scale advertising should only be restricted to the robotics industry. A good way to get the attention of companies would be to ask for a visit so we could pitch the benefits of using our service.

Step 3: What is the expected business profits in the next 5 years?

The first 5 years the enterprise would return little revenue, since the scale of attractiveness has to be created and we have to come to the attention of companies. We can offer little in these 5 years, all income has to be spend on advertisement and hiring data scientists. At the end of the 5 years the enterprise should be able to provide more and better services however, so we can promote the enterprise as an investment in the future.

Step 4: What are the relevant skills and qualifications you have to enable you go into this particular business?

Step 5: What resources would you need to start-up the enterprise and run it consistently for at least one year?

Step 6: How would you go about finding donors and requesting funds?

Papers and summaries

The papers that were read and summarized fit into three categories, namely: remote physical therapy, robot appearance, and robot physics. Below are the three categories together with their respective papers for each team member:

1. Remote physical therapy

Mihail Tifrea

Mishra, A.K., Skubic, M., & Abbott, C. (2015). Development and preliminary validation of an interactive remote physical therapy system. 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 190-193.

2. Robot appearance

Bart Bronsgeest

Bartneck, C. et al. (2009). My Robotic Doppelgänger - A Critical Look at the Uncanny Valley

Abubshait, A. & Wiese, E. (2017). You Look Human, But Act Like a Machine: Agent Appearance and Behavior Modulate Different Aspects of Human–Robot Interaction.

Breazeal, C. & Scassellatie, B. (2002). Robots That Imitate Humans.

Pantic, M. et al. (2007). Human Computing and Machine Understanding of Human Behavior: A Survey.

Adams, B. et al. (2000). Humanoid Robots: A New Kind of Tool.

Jeroen Sies

Bar-Cohen, Y., & Breazeal, C., (2003). Biologically inspired intelligent robots.

Hashimoto, M., & Yokogawa, C., (2006). Development and Control of a Face Robot Imitating Human Muscular Structures

Nakazawa, A., & Nakaoka, S., & Ikeuchi, K., & Yokoi, K., (2002). Imitating Human Dance Motions through Motion Structure Analysis

Doering, M., & Glas, D., & Ishiguro, H., (2019). Modeling Interaction Structure for Robot Imitation Learning of Human Social Behavior

Siegel, M., & Breazeal, C., & Norton, M., (2009). Persuasive Robots: The Influence of Robot Gender on Human Behavior

Goetz, J., & Kiesler, S., & Powers, A., (2003). Matching Robot Appearance and Behavior to Tasks to Improve Human-Robot Cooperation

3. Robot physics

Mihail Tifrea

R. Van Ham, B. Vanderborght, M. Van Damme, B. Verrelst and D. Lefeber (2006). "MACCEPA: the mechanically adjustable compliance and controllable equilibrium position actuator for 'controlled passive walking'