PRE2018 1 Group3 1024503: Difference between revisions

From Control Systems Technology Group
Jump to navigation Jump to search
mNo edit summary
(Moved over State of the Art and reworked it slightly (still in progress))
Line 2: Line 2:
=State of the Art=
=State of the Art=


The use of robots in a retail environment like stores and supermarkets has been a frequent subject for research. This research can be subdivided into a variety of smaller fields:
==Navigation of robots in indoor environments with people and/or other robots==
Navigation of the robot in a shopping environment is an essential function it should have. Several approaches to navigation were found:
Articles here describe the use of models that describe buildings in stores, rooms, a set of places in each room and connectors among these. There is made use of so-called ‘highways’ for pre-determined robot paths and ‘off-roads’ where the robot plans its own path <ref>Gert L. Andersen, Anders C. Christensen, Ole Ravn, Mobile Robot Navigation In Indoor Environments Using Highways And Off-roads, Institute for Automation. bldg. 326, Technical University of Denmark</ref>. A different approach is the use of a sensor space as elaborated on in <ref>Narongdech Keeratipranon, Robot Navigation in Sensor Space, Faculty of Information Technology Queensland University of Technology</ref>.
Other articles describe a modelling approach that predicts the surrounding pedestrian’s actions so that the robot can develop its own path. One promising example is an agent-based modelling approach where surrounding pedestrians are assigned the behaviours interact, watch, curious, ignore, cautious and avoid <ref>Usher, J. M., McCool, R., Strawderman, L., Carruth, D. W., Bethel, C. L., & May, D. C. (2017). Simulation modeling of pedestrian behavior in the presence of unmanned mobile robots. Simul. Modell. Pract. Theory, 75, 96–112. doi: 10.1016/j.simpat.2017.03.012</ref>.
==Design of appropriate actuators==
The robot should be able to move around various objects, these could be heavy or fragile. Article <ref>Lauzier, N. (2018, September 04). Robot Gripper Force Control. Retrieved from https://blog.robotiq.com/bid/53319/Robot-Gripper-Force-Control</ref> describes the use of a force control parameter for robot grippers, so that fragile objects will not be damage by actuators. Article <ref>Michael Zinn, Bernard Roth, Oussama Khatib J. Kenneth Salisbury. A New Actuation  Mastrogiovanni, F., & Casalino, G. (2018). Flexible human-robot cooperation models for assisted shop-floor tasks.Approach for Human Friendly Robot Design. Department of Mechanical Engineering
Stanford University</ref> emphasises the challenge of designing safe actuators for human-centred robotics. The articles states that by reducing the effective impedance while maintain high frequency torque capability in actuators, safety and performance requirements can be achieved.
==Design of appropriate sensors==
Various sensors are needed, especially for localisation and navigation purposes but also for object recognition. According to <ref>Payá, L., Gil, A., & Reinoso, O. (2017). A state-of-the-art review on mapping and localization of mobile robots using omnidirectional vision sensors. Journal of Sensors, 2017. DOI: 10.1155/2017/3497650
</ref> the advances in computer vision have led to an increase in the use of cameras as sensors. They are often combined with other sensors such as odometry or lasers. Omnidirectional sensors stand out in the richness of information they provide. These sensors, together with robust models of the environment are important for designing an autonomous mobile robot.
==Object recognition (in a shop context)==
(Camera) sensors could be used for object recognition, which is an essential task for this robot application. The robot should be able to distinguish a large variety of shopping goods and should be able to detect if the product is misaligned or missing in the shelves. Article <ref>Makihara, Y., Takizawa, M., Shirai, Y., Makihara, Y., Takizawa, M., Shirai, Y., ...Shimada, N. (2002). Object recognition supported by user interaction for service robots. Object recognition supported by user interaction for service robots, 3, 561–564vol.3. doi: 10.1109/ICPR.2002.1048001</ref> describes a vision system where the user can specify an object the robot has to find and bring. When the recognition result is shown, the user can provide additional information, such as point out mistakes. Article <ref>Kejriwal, N., Garg, S., Kumar, S., Kejriwal, N., Garg, S., & Kumar, S. (2015). Product counting using images with application to robot-based retail stock assessment. 2015 IEEE International Conference on Technologies for Practical Robot Applications (TePRA), 1–6. doi: 10.1109/TePRA.2015.7219676</ref> proposes a novel method for obtaining product count directly from an image using a monocular camera. Article <ref>Agnihotram, G., Vepakomma, N., Trivedi, S., Agnihotram, G., Vepakomma, N., Trivedi, S., ...Kumar, R. (2017). Combination of Advanced Robotics and Computer Vision for Shelf Analytics in a Retail Store. 2017 International Conference on Information Technology (ICIT), 119–124. doi: 10.1109/ICIT.2017.13</ref> describes a patrolling robot that detects misaligned and out of stock products and provides the store associates with alert messages.
==The social or legislation issues that arise when robots enter the workspace==
Robots working alongside humans could pose safety issues as well as open up question on how robots should interact (verbally) with humans. Another problem is that the use of robots could make humans redundant in this field of the job market.
Article <ref>Romeo, J. How Will Robot Store Clerks Disrupt Retail? - Robotics Business Review. (2016, July 26). Retrieved from https://www.roboticsbusinessreview.com/supply-chain/how-will-robot-store-clerks-disrupt-retail</ref> says that retail automation is essential in competitiveness, but could lead to the minimum-wage employees being redundant as the robots are far cheaper. Robot store clerks are likely to be a disruptive force for the retail industry, this article states.
Article <ref>Gonzalez-Jimenez, H. (2018). Taking the fiction out of science fiction: (Self-aware) robots and what they mean for society, retailers and marketers. Futures, 98, 49–56. doi: 10.1016/j.futures.2018.01.004</ref> emphasises more on self-aware robots that become a part of society (including the retail sector) where brands are used as self-expression. Article <ref>Darvish, K., Wanderlingh, F., Bruno, B., Simetti, E., Mastrogiovanni, F., & Casalino, G. (2018). Flexible human–robot cooperation models for assisted shop-floor tasks. Mechatronics, 51, 97–114. doi: 10.1016/j.mechatronics.2018.03.006</ref> describes a means for robots to detect human action to make the cooperation between humans and robots in the workspace more attractive.
==Human-robot interactions during shopping activities==
A robot store clerk should also be able to interact with humans. Humans might want information about a product or want to know where it is located. Article <ref>Bertacchini, F., Bilotta, E., & Pantano, P. (2017). Shopping with a robotic companion. Computers in Human Behavior, 77, 382–395. doi: 10.1016/j.chb.2017.02.064</ref> proposes a robotic shopping companion to help customers in their shopping activities. Furthermore, the robot collects the emotional state of people through social interactions and then use that to influence people’s buying decisions. Article <ref>Lee, S. A., & Liang, Y. (. (2018). Robotic foot-in-the-door: Using sequential-request persuasive strategies in human-robot interaction. Computers in Human Behavior. doi: 10.1016/j.chb.2018.08.026</ref> goes further with investigating ways in which robots can persuade people. This could be applied to the robot store clerk in persuading people to buy a certain product.
Article <ref>Gallé, M., Kynev, E., Monet, N., Gallé, M., Kynev, E., Monet, N., & Legras, C. (2017). Context-aware selection of multi-modal conversational fillers in human-robot dialogues. 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), 317–322. doi: 10.1109/ROMAN.2017.8172320</ref> describes ways in which verbal output of a robot can be made more human-like by introducing context-aware conversational fillers.
==A combination of above fields, applied to a designed robot==
These articles describe a fully working system of a robot working in a retail environment. Especially article <ref>Kumar, S., Sharma, G., Kejriwal, N., Kumar, S., Sharma, G., Kejriwal, N., ...Chauhan, V. K. (2014). Remote retail monitoring and stock assessment using mobile robots. 2014 IEEE International Conference on Technologies for Practical Robot Applications (TePRA), 1–6. doi: 10.1109/TePRA.2014.6869136
</ref> is a great example, where a system is built that automates data collection for surveying and monitoring the shelves. The robot here can monitor shelves autonomously or through tele-operation. It can automatically detect out of stock situations. According to this article it will improve customer satisfaction, as shelve products are filled more frequently. The deployment also would not require modifying the existing store infrastructure and has a short return-on-investment period.
'''Navigation'''
[3] Guizzo, E. Ackerman. E. (2015). iRobot Brings Visual Mapping and Navigation to the Roomba 980. IEEE Spectrum: Technology, Engineering, and Science News. Retrieved from https://spectrum.ieee.org/automaton/robotics/home-robots/irobot-brings-visual-mapping-and-navigation-to-the-roomba-980
'''USE'''
[6] Heinzmann, J., & Zelinsky, A. (2000). Building Human-Friendly Robot Systems. SpringerLink, 305–312. doi: 10.1007/978-1-4471-0765-1_37
[7] Wisskirchen, G., Biacabe, B T. et al. Artificial Intelligence and Robotics and Their Impact on the Workplace, April 2017, IBA Global Employment Institute
'''Sensors'''
[10] Mantha, B. R. K., Menassa, C. C., & Kamat, V. R. (2018). Robotic data collection and simulation for evaluation of building retrofit performance. Autom. Constr., 92, 88–102. doi: 10.1016/j.autcon.2018.03.026
[15] Rajithkumar, B. K., Deepak, G. M., Uma, B. V., Hadimani, B. N., Darshan, A. R., & Kamble, C. R. (2018). Design and Development of Weight Sensors Based Smart Shopping Cart and Rack System for Shopping Malls. Mater. Today:. Proc., 5(4, Part 3), 10814–10820. doi: 10.1016/j.matpr.2017.12.367
'''Store clerk robot implementation (technical)'''
[16] Tomizawa, T., & Ohya, A. (2006, October). Remote shopping robot system,-development of a hand mechanism for grasping fresh foods in a supermarket. In Intelligent Robots and Systems, 2006 IEEE/RSJ International Conference on (pp. 4953-4958). IEEE. DOI: 10.1177/1729881417703569
[18] Cheng, C. H., Chen, C. Y., Liang, J. J., Tsai, T. N., Liu, C. Y., & Li, T. H. S. (2017, September). Design and implementation of prototype service robot for shopping in a supermarket. In Advanced Robotics and Intelligent Systems (ARIS), 2017 International Conference on (pp. 46-51). IEEE. DOI: 10.1109/ARIS.2017.8297181
[23] Lin, T., Baron, M., Hallier, B., Lin, T., Baron, M., Hallier, B., ...Dugan, J. (2016). Design of a low-cost, open-source, humanoid robot companion for large retail spaces. 2016 IEEE Systems and Information Engineering Design Symposium (SIEDS), 66–71. doi: 10.1109/SIEDS.2016.7489329
[24] Kamei, K., Ikeda, T., Kidokoro, H., Kamei, K., Ikeda, T., Kidokoro, H., ...Hagita, N. (2011). Effectiveness of Cooperative Customer Navigation from Robots around a Retail Shop. 2011 IEEE Third International Conference on Privacy, Security, Risk and Trust and 2011 IEEE Third International Conference on Social Computing, 235–241. doi: 10.1109/PASSAT/SocialCom.2011.173


=Conclusion=
=Conclusion=

Revision as of 16:19, 4 October 2018

Introduction

State of the Art

The use of robots in a retail environment like stores and supermarkets has been a frequent subject for research. This research can be subdivided into a variety of smaller fields:

Navigation of robots in indoor environments with people and/or other robots

Navigation of the robot in a shopping environment is an essential function it should have. Several approaches to navigation were found: Articles here describe the use of models that describe buildings in stores, rooms, a set of places in each room and connectors among these. There is made use of so-called ‘highways’ for pre-determined robot paths and ‘off-roads’ where the robot plans its own path [1]. A different approach is the use of a sensor space as elaborated on in [2]. Other articles describe a modelling approach that predicts the surrounding pedestrian’s actions so that the robot can develop its own path. One promising example is an agent-based modelling approach where surrounding pedestrians are assigned the behaviours interact, watch, curious, ignore, cautious and avoid [3].

Design of appropriate actuators

The robot should be able to move around various objects, these could be heavy or fragile. Article [4] describes the use of a force control parameter for robot grippers, so that fragile objects will not be damage by actuators. Article [5] emphasises the challenge of designing safe actuators for human-centred robotics. The articles states that by reducing the effective impedance while maintain high frequency torque capability in actuators, safety and performance requirements can be achieved.

Design of appropriate sensors

Various sensors are needed, especially for localisation and navigation purposes but also for object recognition. According to [6] the advances in computer vision have led to an increase in the use of cameras as sensors. They are often combined with other sensors such as odometry or lasers. Omnidirectional sensors stand out in the richness of information they provide. These sensors, together with robust models of the environment are important for designing an autonomous mobile robot.

Object recognition (in a shop context)

(Camera) sensors could be used for object recognition, which is an essential task for this robot application. The robot should be able to distinguish a large variety of shopping goods and should be able to detect if the product is misaligned or missing in the shelves. Article [7] describes a vision system where the user can specify an object the robot has to find and bring. When the recognition result is shown, the user can provide additional information, such as point out mistakes. Article [8] proposes a novel method for obtaining product count directly from an image using a monocular camera. Article [9] describes a patrolling robot that detects misaligned and out of stock products and provides the store associates with alert messages.

The social or legislation issues that arise when robots enter the workspace

Robots working alongside humans could pose safety issues as well as open up question on how robots should interact (verbally) with humans. Another problem is that the use of robots could make humans redundant in this field of the job market. Article [10] says that retail automation is essential in competitiveness, but could lead to the minimum-wage employees being redundant as the robots are far cheaper. Robot store clerks are likely to be a disruptive force for the retail industry, this article states. Article [11] emphasises more on self-aware robots that become a part of society (including the retail sector) where brands are used as self-expression. Article [12] describes a means for robots to detect human action to make the cooperation between humans and robots in the workspace more attractive.

Human-robot interactions during shopping activities

A robot store clerk should also be able to interact with humans. Humans might want information about a product or want to know where it is located. Article [13] proposes a robotic shopping companion to help customers in their shopping activities. Furthermore, the robot collects the emotional state of people through social interactions and then use that to influence people’s buying decisions. Article [14] goes further with investigating ways in which robots can persuade people. This could be applied to the robot store clerk in persuading people to buy a certain product. Article [15] describes ways in which verbal output of a robot can be made more human-like by introducing context-aware conversational fillers.

A combination of above fields, applied to a designed robot

These articles describe a fully working system of a robot working in a retail environment. Especially article [16] is a great example, where a system is built that automates data collection for surveying and monitoring the shelves. The robot here can monitor shelves autonomously or through tele-operation. It can automatically detect out of stock situations. According to this article it will improve customer satisfaction, as shelve products are filled more frequently. The deployment also would not require modifying the existing store infrastructure and has a short return-on-investment period.


Navigation

[3] Guizzo, E. Ackerman. E. (2015). iRobot Brings Visual Mapping and Navigation to the Roomba 980. IEEE Spectrum: Technology, Engineering, and Science News. Retrieved from https://spectrum.ieee.org/automaton/robotics/home-robots/irobot-brings-visual-mapping-and-navigation-to-the-roomba-980

USE

[6] Heinzmann, J., & Zelinsky, A. (2000). Building Human-Friendly Robot Systems. SpringerLink, 305–312. doi: 10.1007/978-1-4471-0765-1_37

[7] Wisskirchen, G., Biacabe, B T. et al. Artificial Intelligence and Robotics and Their Impact on the Workplace, April 2017, IBA Global Employment Institute

Sensors

[10] Mantha, B. R. K., Menassa, C. C., & Kamat, V. R. (2018). Robotic data collection and simulation for evaluation of building retrofit performance. Autom. Constr., 92, 88–102. doi: 10.1016/j.autcon.2018.03.026

[15] Rajithkumar, B. K., Deepak, G. M., Uma, B. V., Hadimani, B. N., Darshan, A. R., & Kamble, C. R. (2018). Design and Development of Weight Sensors Based Smart Shopping Cart and Rack System for Shopping Malls. Mater. Today:. Proc., 5(4, Part 3), 10814–10820. doi: 10.1016/j.matpr.2017.12.367

Store clerk robot implementation (technical)

[16] Tomizawa, T., & Ohya, A. (2006, October). Remote shopping robot system,-development of a hand mechanism for grasping fresh foods in a supermarket. In Intelligent Robots and Systems, 2006 IEEE/RSJ International Conference on (pp. 4953-4958). IEEE. DOI: 10.1177/1729881417703569

[18] Cheng, C. H., Chen, C. Y., Liang, J. J., Tsai, T. N., Liu, C. Y., & Li, T. H. S. (2017, September). Design and implementation of prototype service robot for shopping in a supermarket. In Advanced Robotics and Intelligent Systems (ARIS), 2017 International Conference on (pp. 46-51). IEEE. DOI: 10.1109/ARIS.2017.8297181

[23] Lin, T., Baron, M., Hallier, B., Lin, T., Baron, M., Hallier, B., ...Dugan, J. (2016). Design of a low-cost, open-source, humanoid robot companion for large retail spaces. 2016 IEEE Systems and Information Engineering Design Symposium (SIEDS), 66–71. doi: 10.1109/SIEDS.2016.7489329

[24] Kamei, K., Ikeda, T., Kidokoro, H., Kamei, K., Ikeda, T., Kidokoro, H., ...Hagita, N. (2011). Effectiveness of Cooperative Customer Navigation from Robots around a Retail Shop. 2011 IEEE Third International Conference on Privacy, Security, Risk and Trust and 2011 IEEE Third International Conference on Social Computing, 235–241. doi: 10.1109/PASSAT/SocialCom.2011.173

Conclusion

Discussion

References

  1. ↑ Gert L. Andersen, Anders C. Christensen, Ole Ravn, Mobile Robot Navigation In Indoor Environments Using Highways And Off-roads, Institute for Automation. bldg. 326, Technical University of Denmark
  2. ↑ Narongdech Keeratipranon, Robot Navigation in Sensor Space, Faculty of Information Technology Queensland University of Technology
  3. ↑ Usher, J. M., McCool, R., Strawderman, L., Carruth, D. W., Bethel, C. L., & May, D. C. (2017). Simulation modeling of pedestrian behavior in the presence of unmanned mobile robots. Simul. Modell. Pract. Theory, 75, 96–112. doi: 10.1016/j.simpat.2017.03.012
  4. ↑ Lauzier, N. (2018, September 04). Robot Gripper Force Control. Retrieved from https://blog.robotiq.com/bid/53319/Robot-Gripper-Force-Control
  5. ↑ Michael Zinn, Bernard Roth, Oussama Khatib J. Kenneth Salisbury. A New Actuation Mastrogiovanni, F., & Casalino, G. (2018). Flexible human-robot cooperation models for assisted shop-floor tasks.Approach for Human Friendly Robot Design. Department of Mechanical Engineering Stanford University
  6. ↑ Payá, L., Gil, A., & Reinoso, O. (2017). A state-of-the-art review on mapping and localization of mobile robots using omnidirectional vision sensors. Journal of Sensors, 2017. DOI: 10.1155/2017/3497650
  7. ↑ Makihara, Y., Takizawa, M., Shirai, Y., Makihara, Y., Takizawa, M., Shirai, Y., ...Shimada, N. (2002). Object recognition supported by user interaction for service robots. Object recognition supported by user interaction for service robots, 3, 561–564vol.3. doi: 10.1109/ICPR.2002.1048001
  8. ↑ Kejriwal, N., Garg, S., Kumar, S., Kejriwal, N., Garg, S., & Kumar, S. (2015). Product counting using images with application to robot-based retail stock assessment. 2015 IEEE International Conference on Technologies for Practical Robot Applications (TePRA), 1–6. doi: 10.1109/TePRA.2015.7219676
  9. ↑ Agnihotram, G., Vepakomma, N., Trivedi, S., Agnihotram, G., Vepakomma, N., Trivedi, S., ...Kumar, R. (2017). Combination of Advanced Robotics and Computer Vision for Shelf Analytics in a Retail Store. 2017 International Conference on Information Technology (ICIT), 119–124. doi: 10.1109/ICIT.2017.13
  10. ↑ Romeo, J. How Will Robot Store Clerks Disrupt Retail? - Robotics Business Review. (2016, July 26). Retrieved from https://www.roboticsbusinessreview.com/supply-chain/how-will-robot-store-clerks-disrupt-retail
  11. ↑ Gonzalez-Jimenez, H. (2018). Taking the fiction out of science fiction: (Self-aware) robots and what they mean for society, retailers and marketers. Futures, 98, 49–56. doi: 10.1016/j.futures.2018.01.004
  12. ↑ Darvish, K., Wanderlingh, F., Bruno, B., Simetti, E., Mastrogiovanni, F., & Casalino, G. (2018). Flexible human–robot cooperation models for assisted shop-floor tasks. Mechatronics, 51, 97–114. doi: 10.1016/j.mechatronics.2018.03.006
  13. ↑ Bertacchini, F., Bilotta, E., & Pantano, P. (2017). Shopping with a robotic companion. Computers in Human Behavior, 77, 382–395. doi: 10.1016/j.chb.2017.02.064
  14. ↑ Lee, S. A., & Liang, Y. (. (2018). Robotic foot-in-the-door: Using sequential-request persuasive strategies in human-robot interaction. Computers in Human Behavior. doi: 10.1016/j.chb.2018.08.026
  15. ↑ Gallé, M., Kynev, E., Monet, N., Gallé, M., Kynev, E., Monet, N., & Legras, C. (2017). Context-aware selection of multi-modal conversational fillers in human-robot dialogues. 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), 317–322. doi: 10.1109/ROMAN.2017.8172320
  16. ↑ Kumar, S., Sharma, G., Kejriwal, N., Kumar, S., Sharma, G., Kejriwal, N., ...Chauhan, V. K. (2014). Remote retail monitoring and stock assessment using mobile robots. 2014 IEEE International Conference on Technologies for Practical Robot Applications (TePRA), 1–6. doi: 10.1109/TePRA.2014.6869136

Planning

Week Milestones
5
  • Pin down exactly what to work on and towards
  • Write up a new planning
  • Make wiki skeleton
6
7
  • Finish up final presentation
8
  • Do final presentation
  • Finalise wiki
Retrieved from "https://dsdwiki.wtb.tue.nl/index.php?title=PRE2018_1_Group3_1024503&oldid=62888"