PRE2018 1 Group3: Difference between revisions

From Control Systems Technology Group
Jump to navigation Jump to search
No edit summary
(Replaced content with 'https://docs.google.com/document/d/1CrYVH1GT-tkQKrk9QaKfZt8_slSX9KY9zquc9tXW82A/edit')
Line 1: Line 1:
0LAUK0 Group 3:
https://docs.google.com/document/d/1CrYVH1GT-tkQKrk9QaKfZt8_slSX9KY9zquc9tXW82A/edit
 
Dirk Heldens            0969145
Geert van Wordragen        1024503
Islam Lakehal
Nosa Dielingen
Tristan de Waard        0996117
Subject:
Store clerk robot
Problem Statement:
Food prices have been steadily growing since 2006 [BRON]. Because of this, the highly competitive supermarkets have had to increase prices tremendously. For example, butter and margarine prices have gone up with a staggering 54.9% over the past 10 years [BRON]. It is important to give supermarkets more chance to compete and increase their customer satisfaction.
A solution might be found by streamlining, increasing the efficiency and economizing the state-of-the-art supermarket. Since self check-out services already are cost efficient, enhance customer satisfaction and are already becoming more and more abundant [BRON], an emphasis will be laid on the role of a store clerk in the supermarket.
This research proposes the design of a robot store-clerk that either assists the human store clerk in inconvenient tasks, or acts as a replacement for the store clerk. Apart from the expected positive economic effects for supermarket enterprises, the negative and positive effects on users and society will be investigated thoroughly as well.
To investigate the user’s needs, two surveys will be conducted: one targeting the general consumer another targeting store clerks.
To demonstrate the functions of a store clerk robot a prototype will be made.
Objectives:
First of all, the basic tasks of a store clerk will be described concisely. Then, it is investigated which tasks can be delegated to a robot looking at state-of-the-art robot technology. A literature study will be performed to examine the financial consequences for a supermarket and the impact on society and users when employing a robot store clerk. Legislation aspects and responsibility regarding a store clerk robot will be considered also. The outcomes of these studies provide several requirements, preferences and constraints for the robot design. The survey targeted to customers and store clerks will be used to define additional design requirements. Next, a robot store clerk design will be proposed based on the previously defined tasks and requirements. Subsequently, a prototype will be built to demonstrate the functioning of a robot store clerk in a test environment and to look at the overall viability of robot store clerks in practice. Finally, a conclusion will be drawn on the desirability of employing a robot store clerk in the long and short term future. The most important user requirements and their accommodation in the design will be listed.
 
Store clerk task description
The job description for a store clerk [BRON] gives the following tasks a store clerk has to do:
Detecting where products are missing in the shelves.
Restocking missing products in shelves with oldest product upfront (First-in-first-out system)
Making sure products and their labels are displayed correctly (Dutch: ‘spiegelen’)
Helping customers find products / answering their product related questions.
Helping out the cashier when products need to be replaced at the counter.
Unloading goods from the storage truck, before the supermarket opens.
Finding products in the storage and bringing them into the supermarket.
 
Possible robot store clerk tasks
Looking at state-of-the-art research papers, the store clerk tasks will now be assessed on their possibility of being carried out by a robot.
First of all, task #1 can according to [20] be done by making use of a patrolling robot either tele-operated or autonomous that scans shelves for misaligned or out-of-stock products. In the case of detection, a real time alarm is sent out so further action can be taken by staff-members or, preferably, other robots. It should be noted that this system can also detect where and when tasks #2 and #3 need to be performed. The entire solution ensures reduced store clerk manpower needed for these task and costs around US$2000.
Task #2 will require a robot with advanced sensors and actuators that can take out a row of products and put the new product in the back. With the current state-of-the-art robot designs this is still a very inefficient task for a robot to do, so at the moment human store clerks will need to perform this task. However, if an entire row of products is labeled out-of-stock by the system described in the previous subsection a robot might be able to fill up the shelf by inserting a row of products. Berntorp. K et al. [BRON] have proposed a design for robot manipulators for use in retail stores but it is only able to move one object at a time. It is therefore thought that this task should still be carried out by human store clerks.
    Task #3 There are already several robots on the market that detect misaligned products, for example Simbe Robotics’ Tally robot  [BRON]. This robot gives staff members an overview of retail statistics, also alerting where products are misaligned. Staff members still have to do the task themselves, as there are no robots on the market that can perform this action.
    Task #4 requires a way customers can interact with the robot. Cheng, C. H. et al. [18] have already proposed a solution for this by designing a graphic user interface where customers can choose from product pictures, so that the robot can guide them to the product’s location or provide product information. Moreover, the customer is also able to give the robot commands or ask questions through speech.
    Task #5 requires the robot to detect the product that is given, and should then be told what replacement is needed. It should then be able to find both product’s locations and replace the products accordingly. This requires advanced sensors and actuators and this is probably not possible as of now, since current actuators are not capable enough.
    Task #6 is possible through the design of a robot, however this design will differ a lot from a store clerk robot, so it should be seen as a different robot. This task might be accomplished with a forklift type robot that can do the basic task of unloading pallets from a storage truck. For low budget supermarkets where most products are presented in pallets rather than shelves this task is especially important, when now the can also perform the restocking task. The design of a pallet lifting robot is given by Garibotto, S. et al. [BRON] However, it should be noted that this design does not incorporate appropriate movement in environments with humans walking around.
    Task #7 could be achieved through a similar approach as task #6 where pallets from the storage are brought into the supermarket so that store clerks can do their restocking tasks afterwards.
 
User
User types:
For the store clerk robot, several users can be identified:
The supermarket boss/manager
They ‘own’ the robot and thus have a certain responsibility over the robot
Supermarket customers
Customers can ask questions about the location of certain products or ask if products are in stock / when they come back in stock.
Supermarket store clerks
In the case that the robot works alongside human store clerks, they become the users as well.
 
The general user requirements for a robot store clerk can be formulated now.
General user requirements
Safe and efficient ground floor robot movement
Both customers and store clerks do not want to be obstructed by the robot’s movement, so the robot should have mechanisms to avoid obstacles and humans. It should move efficiently to reach areas where tasks need to be completed as quickly as possible.   
Efficient refilling of shop shelves
Shop owners strive for a high turnover, so it is important that shelves are in stock.
Careful handling of shopping goods
Inflicting damage to shopping goods renders them unmarketable, which is detrimental for the shop’s turnover and thus undesired for shop owners.
Shop customers must be able to interact verbally with the robot
Shop customers need to be able to gather information about product location or stock.
Low cost maintenance of robot store clerk
The robot should require less maintenance cost than the average salary of a human store clerk.
Low error rate in detection of shopping goods
The robot should correctly detect shopping goods with a low error rate in order for it to do its tasks correctly, and increase customer satisfaction.
 
Now, the benefits of employing a robot store clerk will be discussed.
Store clerk benefits
Since it is most likely not possible in the near future that store clerks will be entirely replaced by robots, a hybrid situation in which store clerks work alongside robot companions is believable in the short-term future. So what are the benefits for store clerks when working with robot companions?
The obvious benefit for store clerks then is that their job will require less tasks when a robot store clerk is introduced in supermarkets. The section above already describes several tasks that can be delegated relatively easy to a robot. Especially the first task is easy to replace and real-world applications already exist. For example, having a robot like Tally would already take out the task of checking the shelves for refilling. This robot gives useful real-time statistics and alerts to store clerks so they can distribute their tasks more efficiently and gives them more time for customer interaction.
Customer benefits
What are supermarket customer benefits when employing a robot store-clerk?
 
Finding a product or location they are headed to, is something customers regularly need help with. This is not only the case in supermarkets but also in e.g. airports or hotels. The good thing is that this a very viable short-term application for robots, as this requires very little robot agility. It is evident that in order for this to be a benefit, user requirement #1 should be taken into account. Darvish, K et al. [13] have proposed a promising solution for this with a very flexible option for robot navigation with obstacle avoidance in a full reactive fashion. Since robots can do this task very efficiently it is thought that customers will benefit from this. Doering, N. et al. [BRON] did a study on customer satisfaction and shopping effectiveness when a robot helped in guiding users to a certain product. It concluded that seniors of 65 years or older were less effective using the product search interface than other sociodemographic groups. However, the overall customer satisfaction turned out high and the participants’ intention to use the robot in the future was high.
Another benefit for customers is that robots can quickly access goods that are out of reach or stored behind glass. Customers no longer have to wait for a free member of supermarket staff, thus saving time. An example of this is seen in Best Buy’s Chloe robot. [BRON] Here, customers can choose the product they would like to buy and the robot arm grabs it for them.
Society:
Loss of jobs
The most obvious consequence of any kind of automation is a loss of jobs. Since these robots will take over some part of the store clerk work, this will not be any different. Store clerkship is a popular part time job for teenagers, employing 11% of the Dutch part time working youth in 2015. [BRON]
Loss of social contact
Supermarkets are, in a way, a social place as well. Automating tasks such as helping customers find the products they need will reduce social contact [citation needed?] and loss of social contact can noticeably affect health. [BRON] The store clerks will also have less human colleagues to talk to.
Cheaper products
One of the main reasons why automation is being used is because it saves the owner money. While, as mentioned before, some of these savings are to the detriment of otherwise potential store clerks, they can also benefit society as a whole.
 
https://www.forbes.com/sites/nikkibaird/2018/06/19/robots-automation-and-retail-not-so-cut-and-dried/#3092eacf7b06
https://www.strategy-business.com/article/Why-AI-Will-Create-Jobs?gko=c2a84&utm_source=itw&utm_medium=20180531&utm_campaign=resp
Enterprise
 
The relevant enterprises are the supermarkets that the store clerk-helpers get introduced to. For these supermarkets their benefit from these robots should be an increase in profit. This can happen both from saving money by needing to hire less store clerks and by getting more customers because of better service provided by these robots.
 
 
 
Streamlining a supermarket:
 
Increasing efficiency:
A robot will be able to perform some tasks a lot faster than humans so letting it do these will make things generally be done quicker, this can make the store more attractive to customers. Decreasing the amount of humans working there will also make a store easier to manage because a robot does not need to be told what to do and is always available.
 
Economizing:
Less store clerks need to be paid for
 
When the store clerks are assisted by a robot they can work more efficiently so this would mean less store clerks have to be hired to do the same amount of work. The money saved on their wages is equal to their hourly wage. In the Netherlands this is equal to €2.50-€12.00 per hour per clerk. [bron] The money saved should compensate for the cost of maintaining the robot and the power used. The increase in profit should eventually make up the for the cost of purchasing the robot, how long this takes depends both on the cost of the robot and how much labor it saves.
 
Effects on competition:
    New technology is already providing different ways to shop for groceries. Home delivery has started to gain traction: “the online grocery shopping market has taken until now to gain traction, currently constituting 6.9% of the UK’s grocery market, but projected to increase 68.3% to £17.6 bn by 2021.”  [bron]. In order To make investing in this robot profitable traditional supermarkets need to keep existing for some time, if supermarkets get phased out in favor of  home delivery it would mean it might be better to not invest too much into their future. [bron] [bron] It is also worth considering that supermarkets may become smaller because of decrease in demand so the robot could be designed with this in mind. The extra convenience that a robot can provide could also help traditional supermarkets compete with the increasing competition.
 
Is this technology still viable in the long-term future when supermarkets might be replaced by magazines with home delivery?
 
Freeing up workspace with for example Best Buy’s Chloe robot
 
Using retail statistics to improve turnover with for example Tally robot
 
The disadvantages
Legislation & responsibility
Survey
For customers
    zie apart docs [1] [2]
 
For store clerks
    Cooperation with robots, would you want that?
    Which tasks are the most tedious?
 
State of the art
25 papers (Full adaption of the robot replacing clerks, with a cohesive system that manages the store’s storage)
 
Navigation
[1] Gert L. Andersen, Anders C. Christensen, Ole Ravn, Mobile Robot Navigation In Indoor Environments Using Highways And Off-roads, Institute for Automation. bldg. 326, Technical University of Denmark
 
[2] Narongdech Keeratipranon, Robot Navigation in Sensor Space, Faculty of Information Technology Queensland University of Technology
 
[3] Guizzo, E. Ackerman. E. (2015). iRobot Brings Visual Mapping and Navigation to the Roomba 980. IEEE Spectrum: Technology, Engineering, and Science News. Retrieved from https://spectrum.ieee.org/automaton/robotics/home-robots/irobot-brings-visual-mapping-and-navigation-to-the-roomba-980
 
[12] Usher, J. M., McCool, R., Strawderman, L., Carruth, D. W., Bethel, C. L., & May, D. C. (2017). Simulation modeling of pedestrian behavior in the presence of unmanned mobile robots. Simul. Modell. Pract. Theory, 75, 96–112. doi: 10.1016/j.simpat.2017.03.012
 
[17] Payá, L., Gil, A., & Reinoso, O. (2017). A state-of-the-art review on mapping and localization of mobile robots using omnidirectional vision sensors. Journal of Sensors, 2017. DOI: 10.1155/2017/3497650
 
 
Actuators
[4] Lauzier, N. (2018, September 04). Robot Gripper Force Control. Retrieved from https://blog.robotiq.com/bid/53319/Robot-Gripper-Force-Control
 
[5] Michael Zinn, Bernard Roth, Oussama Khatib J. Kenneth Salisbury. A New Actuation  Mastrogiovanni, F., & Casalino, G. (2018). Flexible human-robot cooperation models for assisted shop-floor tasks.Approach for Human Friendly Robot Design. Department of Mechanical Engineering
Stanford University
 
USE
[6] Heinzmann, J., & Zelinsky, A. (2000). Building Human-Friendly Robot Systems. SpringerLink, 305–312. doi: 10.1007/978-1-4471-0765-1_37
 
[7] Wisskirchen, G., Biacabe, B T. et al. Artificial Intelligence and Robotics and Their Impact on the Workplace, April 2017, IBA Global Employment Institute
 
[8] Romeo, J. How Will Robot Store Clerks Disrupt Retail? - Robotics Business Review. (2016, July 26). Retrieved from https://www.roboticsbusinessreview.com/supply-chain/how-will-robot-store-clerks-disrupt-retail
 
[11] Gonzalez-Jimenez, H. (2018). Taking the fiction out of science fiction: (Self-aware) robots and what they mean for society, retailers and marketers. Futures, 98, 49–56. doi: 10.1016/j.futures.2018.01.004
 
[13] Darvish, K., Wanderlingh, F., Bruno, B., Simetti, E., Mastrogiovanni, F., & Casalino, G. (2018). Flexible human–robot cooperation models for assisted shop-floor tasks. Mechatronics, 51, 97–114. doi: 10.1016/j.mechatronics.2018.03.006
 
Human Interaction
[9] Bertacchini, F., Bilotta, E., & Pantano, P. (2017). Shopping with a robotic companion. Computers in Human Behavior, 77, 382–395. doi: 10.1016/j.chb.2017.02.064
 
[14] Lee, S. A., & Liang, Y. (. (2018). Robotic foot-in-the-door: Using sequential-request persuasive strategies in human-robot interaction. Computers in Human Behavior. doi: 10.1016/j.chb.2018.08.026
 
[22] Gallé, M., Kynev, E., Monet, N., Gallé, M., Kynev, E., Monet, N., & Legras, C. (2017). Context-aware selection of multi-modal conversational fillers in human-robot dialogues. 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), 317–322. doi: 10.1109/ROMAN.2017.8172320
 
Sensors
[10] Mantha, B. R. K., Menassa, C. C., & Kamat, V. R. (2018). Robotic data collection and simulation for evaluation of building retrofit performance. Autom. Constr., 92, 88–102. doi: 10.1016/j.autcon.2018.03.026
 
[15] Rajithkumar, B. K., Deepak, G. M., Uma, B. V., Hadimani, B. N., Darshan, A. R., & Kamble, C. R. (2018). Design and Development of Weight Sensors Based Smart Shopping Cart and Rack System for Shopping Malls. Mater. Today:. Proc., 5(4, Part 3), 10814–10820. doi: 10.1016/j.matpr.2017.12.367
 
Vakkenvuller-robot (techniek)
[16] Tomizawa, T., & Ohya, A. (2006, October). Remote shopping robot system,-development of a hand mechanism for grasping fresh foods in a supermarket. In Intelligent Robots and Systems, 2006 IEEE/RSJ International Conference on (pp. 4953-4958). IEEE. DOI: 10.1177/1729881417703569
 
[18] Cheng, C. H., Chen, C. Y., Liang, J. J., Tsai, T. N., Liu, C. Y., & Li, T. H. S. (2017, September). Design and implementation of prototype service robot for shopping in a supermarket. In Advanced Robotics and Intelligent Systems (ARIS), 2017 International Conference on (pp. 46-51). IEEE. DOI: 10.1109/ARIS.2017.8297181
 
[20] Kumar, S., Sharma, G., Kejriwal, N., Kumar, S., Sharma, G., Kejriwal, N., ...Chauhan, V. K. (2014). Remote retail monitoring and stock assessment using mobile robots. 2014 IEEE International Conference on Technologies for Practical Robot Applications (TePRA), 1–6. doi: 10.1109/TePRA.2014.6869136
 
[23] Lin, T., Baron, M., Hallier, B., Lin, T., Baron, M., Hallier, B., ...Dugan, J. (2016). Design of a low-cost, open-source, humanoid robot companion for large retail spaces. 2016 IEEE Systems and Information Engineering Design Symposium (SIEDS), 66–71. doi: 10.1109/SIEDS.2016.7489329
 
[24] Kamei, K., Ikeda, T., Kidokoro, H., Kamei, K., Ikeda, T., Kidokoro, H., ...Hagita, N. (2011). Effectiveness of Cooperative Customer Navigation from Robots around a Retail Shop. 2011 IEEE Third International Conference on Privacy, Security, Risk and Trust and 2011 IEEE Third International Conference on Social Computing, 235–241. doi: 10.1109/PASSAT/SocialCom.2011.173
 
Object recognition
[19] Makihara, Y., Takizawa, M., Shirai, Y., Makihara, Y., Takizawa, M., Shirai, Y., ...Shimada, N. (2002). Object recognition supported by user interaction for service robots. Object recognition supported by user interaction for service robots, 3, 561–564vol.3. doi: 10.1109/ICPR.2002.1048001
 
[21] Kejriwal, N., Garg, S., Kumar, S., Kejriwal, N., Garg, S., & Kumar, S. (2015). Product counting using images with application to robot-based retail stock assessment. 2015 IEEE International Conference on Technologies for Practical Robot Applications (TePRA), 1–6. doi: 10.1109/TePRA.2015.7219676
 
[25] Agnihotram, G., Vepakomma, N., Trivedi, S., Agnihotram, G., Vepakomma, N., Trivedi, S., ...Kumar, R. (2017). Combination of Advanced Robotics and Computer Vision for Shelf Analytics in a Retail Store. 2017 International Conference on Information Technology (ICIT), 119–124. doi: 10.1109/ICIT.2017.13
 
 
State of the Art:
The use of robots in a retail environment like stores and supermarkets has been a frequent subject for research. This research can be subdivided into a variety of smaller fields:
Navigation of robots in indoor environments with people and/or other robots. (e.g. [1], [2], [3], [12], [17])
 
Navigation of the robot in a shopping environment is an essential function it should have. Several approaches to navigation were found:
Articles here describe the use of models that describe buildings in storeys, rooms, a set of places in each room and connectors among these. There is made use of so-called ‘highways’ for pre-determined robot paths and ‘off-roads’ where the robot plans its own path [1]. A different approach is the use of a sensor space as elaborated on in [2].
Other articles describe a modeling approach that predicts the surrounding pedestrian’s actions so that the robot can develop its own path. One promising example is an agent-based modeling approach where surrounding pedestrians are assigned the behaviours interact, watch, curious, ignore, cautious and avoid [12].
 
Design of appropriate actuators. (e.g. [4], [5])
 
The robot should be able to move around various objects, these could be heavy or fragile. Article [4] describes the use of a force control parameter for robot grippers, so that fragile objects will not be damage by actuators. Article [5] emphasises the challenge of designing safe actuators for human-centred robotics. The articles states that by reducing the effective impedance while maintain high frequency torque capability in actuators, safety and performance requirements can be achieved.
 
Design of appropriate sensors. (e.g. [10], [15], [17])
 
Various sensors are needed, especially for localisation and navigation purposes but also for object recognition. According to [17] the advances in computer vision have led to an increase in the use of cameras as sensors. They are often combined with other sensors such as odometry or lasers. Omnidirectional sensors stand out in the richness of information they provide. These sensors, together with robust models of the environment are important for designing an autonomous mobile robot.
 
Object recognition (in a shop context). (e.g. [19], [21], [25])
 
(Camera) sensors could be used for object recognition, which is an essential task for this robot application. The robot should be able to distinguish a large variety of shopping goods and should be able to detect if the product is misaligned or missing in the shelves. Article [19] describes a vision system where the user can specify an object the robot has to find and bring. When the recognition result is shown, the user can provide additional information, such as point out mistakes. Article [21] proposes a novel method for obtaining product count directly from an image using a monocular camera. Article [25] describes a patrolling robot that detects misaligned and out of stock products and provides the store associates with alert messages.
 
The social or legislation issues that arise when robots enter the workspace. (e.g. [6], [7], [8], [11], [13])
 
Robots working alongside humans could pose safety issues as well as open up question on how robots should interact (verbally) with humans. Another problem is that the use of robots could make humans redundant in this field of the job market.
Article [8] says that retail automation is essential in competitiveness, but could lead to the minimum-wage employees being redundant as the robots are far cheaper. Robot store clerks are likely to be a disruptive force for the retail industry, this article states.
Article [11] emphasises more on self-aware robots that become a part of society (including the retail sector) where brands are used as self-expression. Article [13] describes a means for robots to detect human action to make the cooperation between humans and robots in the workspace more attractive.
 
Human-robot interactions during shopping activities. (e.g. [9], [14], [22])
 
A robot store clerk should also be able to interact with humans. Humans might want information about a product or want to know where it is located. Article [9] proposes a robotic shopping companion to help customers in their shopping activities. Furthermore, the robot collects the emotional state of people through social interactions and then use that to influence people’s buying decisions. Article [14] goes further with investigating ways in which robots can persuade people. This could be applied to the robot store clerk in persuading people to buy a certain product.
Article [22] describes ways in which verbal output of a robot can be made more human-like by introducing context-aware conversational fillers.
 
A combination of above fields, applied to a designed robot. (e.g. [16], [18], [20], [23], [24])
 
These articles describe a fully working system of a robot working in a retail environment. Especially article [20] is a great example, where a system is built that automates data collection for surveying and monitoring the shelves. The robot here can monitor shelves autonomously or through tele-operation. It can automatically detect out of stock situations. According to this article it will improve customer satisfaction, as shelve products are filled more frequently. The deployment also would not require modifying the existing store infrastructure and has a short return-on-investment period.
 
�
Approach
Brainstorming, Literature study, Documenting progress, Investigate technical, ethical and legal/financial aspects, making a prototype to simulate real events, document and finalizing wiki, video recording as a presentation.
 
Milestones:
Collect 25 relevant papers
Designate roles
Letting average consumers do a survey
Weekly updates of the wiki
Design prototype
Navigation
Shelf filling function
Consumer support system
Finalize prototype
Survey for shop owners to fill in, to get to know the users opinion on the possibilities.
Make video of the prototype for the presentation
Gaining the opinions of shop owners on the finalized product
 
Deliverables:
25 Papers
Regular updates on the wiki
Surveys
Prototype
Video
 
Roles:
Islam:
Nosa:
Tristan:
Geert:
Dirk:
 
Possible roles:
Updating Wiki
Investigate technical, ethical and legal/financial aspects
Managing meetings and room reservations
Making and testing prototype (documenting progress)
Demonstrating the prototype
Supply resources for the prototype
Making and taking surveys
Record & edit video
Doing the presentation�
Design
Navigatie
Relatief
Checkpoint
Kompas
Camera
Vakkenvullen
Dozen
Grijper
Groot bereik
Camera
Klanten
 
Benodigdheden
Kompas Sensor
Camera
Acceleration Sensor
Grijpers:
Servos
Grip
Tandheugel
Druksensor
Wielen
RP
Microfoon/Display

Revision as of 17:02, 16 September 2018

https://docs.google.com/document/d/1CrYVH1GT-tkQKrk9QaKfZt8_slSX9KY9zquc9tXW82A/edit

Retrieved from "https://dsdwiki.wtb.tue.nl/index.php?title=PRE2018_1_Group3&oldid=62650"