PRE2016 3 Groep7

Welcome to the wiki page of group 7, where we present the development of our project: the intelligent G.U.A.R.D robot. The result of the 8 weeks of our group are shown here, such as the planning, the research topics (e.g. literature studies), the design, and the simulation. Feel free to take a look around!

Group Members

-0895113 Vitto Kamycki (Applied Physics)

-0965862 Jeroen Luiken (Electrical Engineering)

-0899885 Guus Reefman (Applied Physics)

-0931226 Wouter Slot (Applied Physics)

-0896767 Marvin van Tilburg (Applied Physics)

-0944209 Sebastiaan Verhoek (Electrical Engineering)

Introduction

Subject

After the brainstorming session in week 1, we came up with a couple of application fields that we could direct our project towards. These included ‘Smart homes’, ‘Guidance robots’ and ‘Cleaning’. Out of these fields we chose the topic of ‘Guidance robots’. We chose this idea because we thought it was the most meaningful project we came up with. It is also a subject that provokes quite a bit of controversy because of the safety/privacy debate. We saw it as a challenge to create a well-functioning robot that improves safety while trying to not constrain privacy. Now it was time to find a specification within this field, and then we thought of the lack of safety on the streets, that is often highlighted in TV-programs such as ‘Opsporing verzocht’. In the inner city of Eindhoven alone each year 82 per 1000 citizens are a target of street crimes. While streets and buildings are full of cameras in the current day and age, the obtained evidence of criminal actions is often lacking, due to e.g. a too low camera resolution, a wrong mounting angle of the camera, and/or the criminals themselves, that often demolish the cameras during the crime. Could this be improved by using robot technology? We think that is possible, and present the GUARD: the Guidance Unit And Route Director. The GUARD is a robot that is mainly intended for escorting people that feel unsafe during night time. It can guide people to e.g. their home, but its main function is to protect them, for example by calling for help (to 112/911) or by planning routes that avoid less safe streets in a city. This robot could be designed with varying specifications, tailored to the exact functionality that is desirable in a given application area (e.g. a metropole, a city or a small village).

Objectives

The objective of our project is to improve safety and accessibility of living environment. This will be done by creating a concept for a guidance robot in order to help people to quickly and safely reach their destinations. Individuals or small groups that feel unsafe or require additional support to reach their destination can request the help of such a guidance robot.

In order to complete the tasks the guidance robot needs to be able to:

- move in order to guide people

- interact with people

- path find through real life real complex and unpredictable paths.

- find optimal routes for clients to guide them to their destination quickly and unharmed.

- call for support when a dangerous situation occurs.

Optimal design choices will need to be made for each of these points.

Approach

Our solution to the problem of unsafe streets and possible dangers at night is the use of a Robotic system that will guide its users to a safe area, for example their homes. This system has been dubbed the GUARD-system. (Guidance Unit And Route Director). This system consists primarily of Robotic ‘bodyguards’. These robots can either wait at places where they are most likely needed, patrol around areas or be called upon by using a smartphone app. It is then possible to state your destination to the robot, which will then accompany you to that destination. To create the feeling of safety, it will have practical options such as a connection to the local police station. To implement the GUARD-System we need a robot with integrated area knowledge and pathfinding, as well as camera’s and motion sensors to avoid it from bumping into obstacles. It also needs to be stable and rigid, so users can support themselves if needed. Also important is an internet/phone connection for calling aid when needed. Some voice recognition is needed to optimize the system, this is useful for receiving directions and recognizing danger, for example. Since streets are less safe at night, the GUARD-system will primarily be used then. At daytime, the robots can either be stored away and charge, or they can be used as city guides in touristic areas.

Design requirements

There are a couple of requirements for the GUARD robot that it needs to fulfil in order to properly function while maximizing performance and minimizing costs. The robot needs to be able to traverse the cities it is stationed at, at least as fast as a human can walk around, in any sort of weather. It needs to be able overcome all sorts of inclines, but we’ll go for an upper limit of about 30 degrees. This is much more than will be necessary for the cities in the Netherlands, but we would like to future proof our design for other cities with much steeper streets such as San Francisco for example.

The robot also needs to be fairly sturdy and durable, as it will need to run nearly 24/7 in the city while enduring possible abuse from the numerous user in the course of the day. A possibility also exists to implement a capability to support incapacitated people (i.e. drunk people) while escorting them to their destination. It also needs a camera that can view in all directions.

There several different methods of robot locomotion. We feel that due to the cost effectiveness requirement, multiple legs or having snake-like slithering movements should not be used. A drone can be relatively cost effective because it can be used without making any significant alterations to the design and because it can be used to circumvent the problems with the environment. However a wheeled robot is very effective energy wise and will be mainly used in city centers, which are often very wheel friendly. Raising and lowering sets of wheels could be used to get on sidewalks and can keep the robot straight should it come across an incline.

The design also needs to be user friendly in the sense that it shouldn’t be able to hurt the user, even on accident. No sharp edges should be used in the creation of the outer chassis. To prevent it from toppling over, the center of mass should be suitably low to the ground. A lower base or a wide set of wheels can be used to help with the stability of the robot.

Tourists should also be able to interact with the robot without using a smartphone app. This can be done using mostly voice commands but a manual input mechanism might be helpful as a backup as it is possible not all languages are correctly implemented or in case something goes wrong with the voice commands.

Ignoring potential difficulties with the law will make it possible to implement more features that might be more beneficial to the users. It’s possible this way to deal with criminals in a much more direct way. The robot might interfere with a potential criminal using tools such a taser or pepper spray built into it. It might also move in between the criminal and victim and issue some kind of warning. Having the criminal be aware that they’re being filmed might encourage them to leave without doing anything.

But before the robot engages in any kind of conflict, it should be sure that its target is actually a criminal. That’s why advanced face recognition technology could be used to identify people that have committed a crime in the past. The robot would try to avoid routes where such individuals walk around. Seeing as the robots are planned to be city property, it should be possible to connect them to a police database.

While the robot’s priority will obviously be the safety of the client, it should also try to prevent doing unnecessary harm to an assailant. Fitting any sort of guns on the unit would probably only serve to increase the chance of fatalities during an encounter with a criminal. As said before, a taser or pepper spray seems to be the most useful in this regard. However, even these methods can cause fatalities, especially to older people or those with medical conditions. A way for the robot to access some basic medical information when it recognises a face should allow the robot to better take certain decisions in order to prevent any serious injury to a potential assailant.

Different users have all sorts of preferences and seeing as one of our primary goals is helping the user, we aim to offer the user as much choice as possible without needlessly delaying them. There won’t be any choices concerning the design of the robot, as the robots will most likely only be owned by the city that puts them to use. Where the user can make a choice is with all the different settings the robot can offer up to the user.

To make sure the user makes informed decisions about their route, the robot needs to give enough information to user. But too much information is also not a good idea because that might cause people to get needlessly anxious about their route, or start to avoid certain streets because of their danger level. The robot should also be able to adjust its route while escorting a person. This might be done by having it follow the user should it deviate from the path.

State of the art

Danger recognition

One of the important technologies on our robot is being able to recognize threats to the escorted client. One main technology which has this are the so-called “smart” CCTVs.

This technology is not yet widely spread. Smart CCTV refers to visual surveillance systems that analyse and interpret video footage by using pattern recognition technologies. This is even so far developed some cameras can notice violence. The CCTV can recognize this through a person’s irrational or random movement usually seen through violence. This is important for GUARD, the escort robot, as it needs to independently notice threats and then notify local authorities.

A technology can also be used to incorporate existing CCTV-networks into our design. This is called a System of Systems. System of systems (SoS) is a network of complex autonomous systems which interchange information. Several aspects are important to make the SoS work.

-Interoperability: the agents in the system should work separately and autonomous. Each system in the SoS should be able to distribute information to all other systems and all systems must be able to interact autonomously inside and outside the system.

-Integration: each agent should be able to control the SoS and communicate not depending on their characteristics, like software or position. This is the reason why in a SoS all systems need to talk in the same language. Else two agents might not be able to interact directly.

-Autonomy: as said before each agent should act independently and be able to fulfil its task needing no assistance, outside the extra information gained through the SoS.

To incorporate this, a central language between the robots and sensor networks needs to be found. A way with which a threat is separated from a non-threat needs to be obtained from another technology, such as the “smart CCTVs”.

One thing GUARD needs to detect is violence. There are some algorithms which do this exactly. These use a database of sounds and videos of fights, gunshots, blood and explosions. As these are all signs of danger. This is a form of action recognition. Sadly violence detection is one of the least studied versions of this. Enrique Bermejo Nievas et al. (2011) propose that using a database will bring great results detecting violence in certain situations. They wanted to detect violence and created a database of 1000 ice hockey videos which included violence and 1000 action movie fights. They found that a descriptor called MoSIFT, can detect violence in these situations with an accuracy of 90% using the databases. This can be used by our robot. First needed is a huge database of videos of street violence, like beatings, and including a database of sounds which could be sounds of threats or gunfire. This than can be used by a descriptor like MoSIFT or one similar to recognize threats and violence during the escort mission of our robot and thus bringing the client to safety and notifying human authorities.

Security robots

The threat detection is also common in another technology than CCTVs. The security robots. While not extremely developed the security robots show promising ideas which GUARD can use to know when to notify the police and the person it escorts.

Autonomous security systems have great potential. Though there is one main difference between the security robots and GUARD is that regular security robots are mostly deployed when no person may enter and thus can report all noticed intruders. GUARD on the other hand is used on the public streets. It could give a sort of tag to its client but it will still run into many citizens who aren’t a threat and don’t have to be reported. Thus regular heat and movement detection won’t fully cut it for GUARD. A method to further establish how a threat will be defined is needed.

Thus further research would be to also incorporate the danger recognition of the smart CCTVs into the security robots and maybe even combine the different robots and a CCTV system into a SoS. This could greatly improve the efficiency of the GUARD.

A hugely invested in and developed security robot is the Knightscope K5. This is a security robot designed to patrol bigger open areas of companies’ buildings. The company claims this robot can autonomously patrol a certain defined area and can detect anomalies and report these to a central station which is operated by humans. These anomalies are detected using cameras in the infrared range and visual range. It can also pick up wireless signals and thus detect phones and other internet connected devices. Using this it can detect unfamiliar devices which can be a sign of espionage or intruders. The company is now working on gun detection but the company won’t reveal how this will be done. A smaller version, the K3, is a similar model with slightly less features to be used in more enclosed areas.

If the K5 is as good as the Knightscope company claims then this would be a good starting point from which we can base our own design off from. As both GUARD and the K5 are used in open areas, and both need to detect anomalies. Though GUARD needs to better distinguish between threat and friendly as it will be deployed in a larger more public environment.

Autonomously Moving AI

Another important piece of technology the GUARD-robots need is the ability to move autonomously. This technology is a very busy area of innovation, as it is currently a very important area in the use of self-driving cars. The technology described in this paragraph will primarily be based on the current workings of self-driving cars, since most information and innovation resides in this area.

Autonomous moving is needed for the GUARD robots, because the alternative would require remote controlling. At this point one should wonder if robots are needed at all, since then it would be easier to send actual people instead of letting them control the robots from a distance. To create an autonomously moving robot, a few technologies are required. The most important technology is GPS. Without it, the robot is unable to know its location, route and destination. With the GPS-system, the robot only needs to be able to receive inputs from its users to function on a basic level. With these basics, the robot would in theory be able to move the way it is designed to. However, the problem lies in the fact that roads are often uneven and there can be many obstacles in the way (e.g. lamp posts, cars or people). To solve this problem, the robot needs to know its immediate surroundings. This way it can micro-manage its path to avoid collisions. A good example of an existing robot that does that is the soccer robot. This robot uses cameras to find positions of other agents, and estimate the near future of their position by measuring velocities. This technology is very simplified in this example, since it uses basic colours with which the other agents are marked to easily determine positions. But the basic technology certainly exists and looks very useful for our design.

Influence of cameras on crime

To know if the GUARD will help, first there has to be looked at the influence on the next closest technology. The one that is used long enough and widely spread enough is CCTV, or else known as camera surveillance. A study of the effects was done in 1999 in the United Kingdom, the record holder of cameras per capita. This study was done by Coretta Phillips.

The author says the main claimed effects of CCTV were:

1. Caught in the act: catching a crime red-handed could increase chance offenders will be caught and persecuted and thus deterred.

2. Framing: CCTV could reduce crime by deterring away potential offenders who don’t want to be caught on camera

3. Nosyparker effect: reduction of crime could happen because people will use the observed area more, and thus human surveillance will increase as well.

4. Effective Deployment: Better knowledge of risk areas is gained and thus security workers can be better deployed, reducing crime.

5. Publicity (general/specific): As an area is known to be observed criminals will less likely go to this area to perpetrate a felony. As it looks like people take crime more seriously.

6. Time for crime: The longer a crime takes the more likely its caught on camera. Short crime might not be as effected as its longer counterparts.

7. Memory Jogging: publicity encourages potential victims to be more conscious and take precautions.

8. Appeal to Cautious: Those who are more security minded use observed areas more than careless people, thus a displacement of victims is caused.

For our moving robot most effects are of interest. Effects 3 and 7 are less important as our robot moves with our client and does not observe only a certain area. And the robot is the precaution that can be taken and thus already fulfils this role. The author then looked at the actual measured effects of CCTVs on crime. First she looked at property crime in streets. This showed promising results. Areas which had at least two cameras had a noticeable decrease in crime in comparison to unobserved areas. Also no displacement effect was noticed. Overall crime was reduced, This confirms effect 2, cameras do deter potential criminals. At least for property theft, which also includes vandalism and burglaries. The effects faded slightly though for petty thefts and vehicle thefts. In areas like markets the effect was also looked at. This gave also promising results as seen in table 1. This is the effect of CCTVs on property crime in the town centre of Newcastle. This shows that in busier areas the effects are very big. These big numbers are attributed to effects 1 and 2.

Table 1. Effects on property crime in Newcastle town centre in a 26-month period after instalment of the CCTV system.

In another town centre in Airdrie, Scotland, similar effects were noticed. In total crime was reduced by 21% in two years. Crimes like housebreaking, theft from motor vehicles and shoplifting, reduced by 48%. Arson and mischief, like graffiti, reduced by 19%. And again no major displacement of crime was seen. An exception is shoplifting which did slightly increased in Glasgow until CCTV was installed there. Thus areas without CCTVs did not experience as great benefits from observed areas. A big plus is the major increase in reported crimes. There was a 116% increase in detections improving effects 1 and 4, caught in the act and effective deployment. And similar number were noticed across the country.

There were some areas like Doncaster where the influence of CCTVs was only 6%. There were major reductions in criminal damage, robbery and theft-person, but no decrease of shoplifting, burglary, violence or drug offences. In a very few areas no real effect was noticed. These are mostly indoor housing projects where most of the crimes were done by the residents. Though there were several problems with this study. It was done in less than a year after instalment and the effects might not have been established. And also residents who should have monitored the CCTVs were neglecting it. Only 14% of the residents said to daily watch the monitored areas and 33% said they don’t want to report a crime in fear of retaliation. This could have had huge impacts on the effects of CCTVs.

Facial recognition

To protect the clients the GUARD can do more than only find correct paths and call help. It can also actively look if people with a criminal record are around. To do this there have to be some changes in current laws.

A facial recognition program works in a certain way. It uses shapes in the image (mouth, eyes and so on) and gives them a certain vector position in the image, to recognise if this person is a certain person A it looks if the known picture of person A has about the same vector positions as the just received image. It also works with so-called vertical and horizontal edge dominance. These are maps of the face in which the parts where edges of shapes are either primarily horizontal or vertical. These kind of mappings can also be used in recognizing faces.

In controlled still images this can be quite accurate with at least an 80% verification rate and maximal a 0.1% false acceptance rate [Overview of the Face Recognition Grand Challenge, P. Jonathon Phillips et al. , 2005]. A problem is that this is only with still images, the person makes the same expression as the sample image, usually a neutral expression, and is at the same distance as the sample image.

In uncontrolled images and 3D-mappings of the face the verification rate is lower and the false acceptance rate higher. While if 3D-mappings are mixed with controlled stills the verification rate is higher than just only controlled stills. In the real world things like bad lighting and moving targets can greatly reduce the effectiveness of the software.

In the regular facial recognition software only one pattern is used as an input. This can be improved by Multi-viewpoint Patterns for Robot Vision suggested by Kazuhiro Fukui and Osamu Yamaguchi in 2005 in Robotics Research. This takes advantage of that the robot is a moving object. This means that the robot can take many images and look at more patterns of the face. This increases the input size of the sample and can improve the verification rate.

To do this kind of input a new method of comparing facial patterns had to be made. The Fukui and Yamaguchi use CMSM (constrained mutual subspace method). This is based upon MSM (mutual subspace method) which uses many inputs instead of the one and tries to find same shapes by looking at canonical angles within the subspace. This subspace is created by a method called the Karhunen-Loève expansion. In the image is shown the difference between regular facial recognition software and MSM. One can easily see that in MSM many more inputs are be used compared to the one single of the regular. This means it can tolerate more differences in facial shapes.

But MSM has a problem, it ignores rival subspaces, which are subspaces which have the same facial structure but aren’t in the same subspace. This is the reason CMSM can improve upon MSM.

CMSM includes an extra subspace on which all rival subspaces are projected. From here on they are compared and looked for similarities. CMSM eventually gets a verification rate of 99% which is an improvement upon MSM with 89% and the standard method with around 80%.

Face structure and criminal behaviour

Outside of regular facial recognition to find people who have a criminal record, the GUARD could go one step further. Is it possible to guess if someone is dangerous by the shapes of one’s face? From a research of Xioalin Wu and Xi Zhang it shown that they made a algorithm which can distinguish between criminal and non-criminal faces. It seemed that criminal faces had more variations in facial appearances than faces of law-abiding citizens. They also noticed certain trends in shapes in criminal faces from slightly bigger lip curvature and certain angles around the nose. This can be used to pick up certain dangerous individuals which the GUARD would try to make the client avoid. They used machine learning asking face recognition software to guess whether a person in an ID-style picture was a criminal or not, and then feeding it the correct answer. A peer review on this research though has major concerns. One is how they acquired the data. Criminal pictures were gained from a Chinese database for photos of criminals whereas the non-criminal pictures were obtained through the internet using profile pictures of Chinese people. This difference in sources of data could explain why the software eventually could differentiate between criminals and non-criminals.

The reviewer Timothy Revell also staked the concern that computers aren’t bias free. The biases of the creators always return into the product. This means that algorithms will make false conclusions while still having the air of legitimacy.

But the reviewer also only says that facial recognition used for general behaviour is inaccurate. One can use it for things like seeing sleep depravity or fear.

This last thing can be used for the GUARD as it can look if someone looks angry, anxious or threatening. Then depending on the facial expression choose to avoid or inform.

USE aspects

The society

The GUARD robot will have the most impact on the society with respect to the other USE aspects. The main goal of this robot is to achieve a safer society whether it is in an urban area, suburban area or more desolate areas. However, right now the main focus is urban areas and inner cities. To be more specific, we use Eindhoven as an example. In the year 2015 Eindhoven was in the top 5 most criminal cities in the Netherlands according to the ‘AD misdaad meter’[1]. The goal of the GUARD robot is to bring this down. The GUARD will be equipped with a camera and walk with the pedestrian, this will offer more personal surveillance and the person guarded by our GUARD robot will always be under watching eyes and if something were to happen the response is only seconds away since the robot will also be equipped with a direct link to the local authorities and the closest hospital if physical harm is done as well. There has been a lot of research whether or not extra sets of eyes on the street have a positive effect on the crime rate. According to Coretta Phillips CCTVs are effective “The paper concludes that CCTV can be effective in deterring property crime, but the findings are more mixed in relation to personal crime, public order offences, and fear of crime.”[2] We believe that implementing the GUARD robot will also affect the other types of crime but mostly personal crime. This would be a great addition to the local police and positive nonviolent solution to reduce crime rates in the society.

The users

When the Users are considered it is clearly visible that the GUARD can be utilized by a large audience. During this project we want to focus on a particular group in order to narrow down and specify the necessities of the GUARD. This constitutes what kind of specifications it requires and what its main purpose is.

We decided to focus on tourists (in touristic cities/areas) as our Users. Since we focused our Users we can now define some necessities of our GUARD robot. Tourists often need a map of the city or area in order to get around and see the most popular attractions, however, they also want to get to their destination safe and sound. This means our GUARD robot needs a build-in navigation system that has up-to-date information about which areas the tourists are statistically most likely to get robbed. The GUARD will be able to provide the safest route for the tourists. However, in the unlikely event that the tourist is being robbed our GUARD can instantly notify the crime to local authorities or if the tourist becomes unwell it can notify the closest hospital and mobilize an E.M.T. unit.

The GUARD can be mobilized in two main ways, it works similar to the taxi system. The tourist can call or use an applet on the phone to mobilize a GUARD robot to its location or the tourist can walk to a certain stop at points that are easily reachable by public transport, this maximizes the utility of our GUARD robot.

Nonlethal intervention

The main purpose of the GUARD is to decrease the amount of personal crime (described in USE society) in mainly big cities. The first method that the GUARD uses is to be a deterrence factor for the criminal. The attacker knows he/she will be recorded and the local authorities will be notified immediately, however, in some cases this might not be enough and we want our user to feel as safe as possible. If the user of the GUARD is being attacked (physically) there needs to be intervention, in most cases the local authority will arrive too late and the attacker might have fled already. While the attacker is now recognized and familiar in the police database as a criminal, the user might still not benefit as much from the GUARD since she/he still got attacked. This can be prevented by demobilizing the attacker by the means of nonlethal weapons on the GUARD. In order to stop the attacker, the GUARD is equipped with a Taser. In the case of a physical attack on the user the GUARD can demobilize the attacker and immediately notify the local authorities so that the attacker can be detained. This method is very well-suited since it is going to reduce the overall crime-rate without causing any deaths.

Legal issues

There are several issues and ethical dilemmas regarding the use of any type of (nonlethal) weapon systems on the GUARD. The main issue that comes into play is when the GUARD uses it weapon when it is not needed. The victim (‘tased’ person) might sue someone since he/she believes it is unnecessary use of violence and in this case someone must be held responsible, but who? Right now it is also not allowed by law to record someone on the streets where their face can be recognized on camera, by law. Article 139f of the Dutch Penal Code explains why this is not allowed. This brings us to another legal issue as of right now since the GUARD is not allowed to record in public and immediately store it in a database according to WBP (Wet Bescherming Persoonsgegevens) (from EU law Data Protection Directive). These two laws are the main laws that prevent us from implementing the GUARD since it stops us already at the issue of the use of a camera.

The foreseeable future

In the future we believe that these laws may be adjusted according to our GUARD since the GUARD is a great solution to reduce the rising crime rates in large cities. It is a proven fact that only the use of CCTVs already deter the criminals (or to-be criminals) which can be found in the section Influence of cameras on crime. Since it is already proven that CCTVs work on reducing property crime and in some cases personal crimes as well, we believe that already due to the fact that the GUARD will provide extra angles. The users will be safer since they now can pick the safest route, have a deterrence factor along the side of them and the attacker can be stopped by the means of nonlethal intervention.

People's opinion on the GUARD

In order to get a better view of what people’s view is on the GUARD and how they think it would look like we conducted a survey. We managed to ask a total of 31 people (mostly family and friends) and we mentioned the general opinion and noticeable trends in the answers. Below you can find the specific questions:

1. In general, what do you think of the GUARD?

2. What are some potential improvements?

3. What’s your opinion on equipping the GUARD with a Taser?

4. What do you think of facial recognition?

5. And lastly, would you mind if you see other people using the GUARD?

When most people first heard about our project they were fairly skeptical because in their eyes they felt like they had to put a lot of trust in the technology of the GUARD. However, after we mentioned the GUARD will not take action without authorization of a human supervisor they did not mind the idea as much. This brings us to the second question since people like the concept but thought some improvements are able to be made. One suggestion that kept coming up was to add a little safe or lockable compartment in the GUARD where one could store valuables, so that if they were to get threatened by an attacker, the attacker would not be able to get to the valuables as easy. Another suggestion was to make it elderly friendly by having some form of support where people using the guard can hold on to. Finally a suggestion was to add a medical kit with a defibrillator that could be used in case of an emergency.

A more controversial subject is the added Taser to the GUARD. When we asked this question people immediately started to imagine everyone getting ‘tased’ without warning. This is obviously not our intention and here it is very important for people that it is known that the GUARD cannot attack without authorization of a human supervisor since the artificial intelligence is clearly not developed enough to handle difficult human situations. After we told them this would be the case, and the GUARD would thus not attack without warning, they were more at ease with the GUARD. Most people did not mind the facial recognition and some of our friends linked to something similar on a phone but then more sophisticated. When we mentioned that in some cases your facial features might be shared with a criminal database most of them said as long as you did nothing wrong you should not mind it. Lastly we asked what you would think if you saw other people use the GUARD. The main response was that at first it would look somewhat ‘awkward’ but eventually it will start to look normal.

Design Choices GUARD

Movement:

There are several options for movement of the GUARD robot. The first major distinction is whether the robot should move over ground or fly in the air, as both have their own advantages and disadvantages. Moving over ground, like humans do, means that the robot needs to be able to move almost like an actual human, e.g. move over all kinds of small bumps, avoid obstacles and other people, climb stairs, etc. This requires a quite flexible way of moving, which can be hard to implement mechanically. On the other hand, the robot will be a lot less vulnerable moving over ground than if it would be flying. It will be less influenced by things like the weather, and it is also more sturdy in general, as a flying drone can be quite easily hit by a another person. If the robot is sturdy enough, it can even be used as a support by (elderly) users. A robot that flies through the air thus has a disadvantage concerning reliability and sturdiness. It can be easily destroyed or even crash, causing potential harm to the people surrounding it. There’s also the problem of battery life. Current drones can’t even fly for more than half an hour before requiring a recharge. Regular robots with wheels are much more efficient in that aspect. A final problem is that legislation on drones currently forbids or heavily restricts their use in urban areas. An advantage of flying is that there are less obstacles to avoid (assuming the robot flies sufficiently high), and that the type of terrain is irrelevant. This goes both ways: the robot does encounter less obstacles, but other people do as well, as the robot is now above, instead of amongst them. Moreover, the robot will have a more clear and stable view of its surroundings, as most of the important objects, such as its client, will be below it. The robot can watch everything from a bird-eye view, although a disadvantage is that images taken from people from above may not be as clear, which can be problematic when an offender’s face should be recognized using these images. A small flying drone may also be a bit cheaper than a bigger robot that moves over ground, but this depends on its features of course. However the robot being more fragile will result in them breaking down faster, causing more maintenance costs. This might make drones more expensive in the long run as costs accumulate.

The final design choice is to make a robot that moves on the ground. The main reasons for this is that a robot moving over ground can be made a lot sturdier. Furthermore, there are less legal problems that have to be taken into account, and the design has less restrictions. For example, the weight is less important, allowing for more features and a much longer battery life. A second major distinction is the exact way in which the robot will move over ground: it could either walk, drive using wheels or drive using tracks. A walking robot might look great and would be able to move wherever a human could also walk, but those kind of robots are still very expensive, complicated, and far from fully functional. Wheels are more flexible and energy efficient, but obstacles may be hard to scale, and some terrain might be difficult to traverse with wheels. Tracks offer practically all the advantages wheels have, with the added benefit of being suited for almost any kind of surface. Tracks can also be used to scale small ledges such as the curb or small steps, and are much more stable due to the increased surface area.

The final choice for movement are tracks. A benefit of tracks is the great stability it gives. Tracks also give great traction when going up an incline, as this is one of the requirements earlier discussed. It is also the easiest solution to curbs. As low curbs can be scaled by tracks, while with wheels that’s much harder. Tracks may be somewhat more intimidating towards the user as opposed to, say a walking robot. But here the choice of practicality outweighed the negative effects a user might experience from its appearance. As not being able to operate in a city would completely obsolete the user feelings.

Sensing:

Vision:

Vision is essential for the GUARD since it is needed for detecting obstacles and routes, checking for undesirable situations, etc. Therefore, the camera might be the most important sensor for the user. This can be seen from the positive influence of cameras on the crime rate and the sense of safety that it gives the user, which are both significant. Although it is important to avoid giving people a feeling that they are watched by ‘Big Brother' as that will then reduce their sense of safety. The most common way to implement vision in a robot is by using cameras. This could be done using either one or multiple cameras, and these cameras could be of different types. The final design choice is to put a set consisting of two cameras on the robot: one regular high quality camera capable of facial recognition and one 360˚-camera which can be used to give the robot a general sense of what’s around it and what to avoid, which can aid in the pathfinding of the robot. This dual camera set will be put on the front of the robot, and the cameras will always be visible. A dual camera is preferred as face recognition using the CMSM method will be easier with more inputs, and being able to judge depth means it can be used for pathfinding too. Furthermore, a single camera could mean that the robot has not enough sight around itself, reducing its performance. The robot itself can’t see everything of course. The resolution of the camera is best set as high as possible, but even the best camera will eventually lead to a loss in detail the further the target is away from the robot. This might cause the robot to notice suspicious people a bit too late. A solution to this is to have all the robots in service share their information between each other. The robots will continue to scan for potentially dangerous people and mark any area that contains them and evaluate the risk of traveling through that area. This kind of analysis could also be partly done by stationary cameras scattered around the city.

The decision not to put cameras everywhere on the robot was made to prevent the earlier mentioned 'Big Brother feeling', as that is a big challenge in this project. Instead, swapping most of these body mounted cameras for a single 360˚-camera will reduce the effect of the 'Big Brother feeling' while still maintaining sight of the robot's surroundings. Also, this option will likely be the most efficient in terms of performance versus cost. The choice to make the cameras visible is to make sure that the client knows they are being watched over. The deterring effects of cameras are also mostly caused by the knowledge of there being cameras, so visible cameras will decrease the chance of being assaulted or robbed. Having visible cameras means also that the GUARD stays within laws here. This is good as the more laws that are broken in the design, the harder it will be to fully realize the robot.

Sound:

Sensing sound will have two main roles in the GUARD, namely voice recognition and recognizing sounds of danger. This means that the GUARD needs to be able to perceive sounds from all sides in good quality. More specific, the GUARD needs at least one sound sensor that allows the user to communicate with it, and one that picks up other sounds. These could be combined into one device of course. The final design choice is to place a noticeable microphone on the GUARD, via which voice commands can be given by its user. This spot needs to be clear, so that users know that there is a spot to talk to when communicating with the GUARD. This is preferred since the GUARD will not be very humanlike, possible causing the user to feel like their voice commands might not be registered by the robot. Having a more dedicated place to give input, with possible feedback via e.g. a light, can lessen that feeling. For the recognition of sounds originating from the surroundings of the robot, extra (less visible) microphones will be placed on and in the GUARD, giving full coverage of sound recognition aspects and allowing the robot to recognize the direction the sound is coming from.

The robot will also benefit from a series of standard voice lines that the robot can say via its speakers. It will mostly be used to simply audibly indicate the route that robot is taking to the user. Some lines could also be used to warn potential criminals that they’re being filmed, should the GUARD detect a potential robbery or assault. Other information concerning their route could also be relayed to the user like this.

Touch:

Touch will most likely not have any added value to the GUARD robot since vision should be able to do most of the sensing. It could be used to detect the robot being stuck against objects but as mentioned previously vision should be able to do the same. The final design choice therefore is to not implement touch sensors, as they would have no added benefits, but would increase complexity and costs.

Power:

The GUARD needs to be able to bring people to their destination and thus must have a large enough battery life to be able to travel that far. Preferably the GUARD is able to drive around for an even longer time than that so new clients can be helped quickly. Having enough recharge stations around the city and having a reasonable amount of spare robots that can fill in for the recharging robots will lower the battery life needed to function properly.

This means that battery life will become a balance between utility and costs. Ideally the GUARD will be able to drive for extended periods of time when it is most likely to be used which will likely be between 22.00 and 3.00. For drones this battery life will probably not be achievable since the drones need to be light. Commercial drones can currently reach a maximum of 25 minutes flight time before needing a recharge, which will make its implementation in the city hard and expensive. Since the final design is not a drone, it can be equipped with larger batteries to increase its range. As the required capacity of the batteries depends on all the other hardware that is implemented, a final design choice was not made, but a battery life of 1 hour is a minimum requirement to allow the GUARD to function properly.

Charging:

In order to keep the GUARD operational as long as possible the GUARD needs to be able to charge quickly. One way to charge the GUARD is by having charging stations distributed throughout the area. These charging stations can also be used as central pick-up-points for the GUARD. The GUARD is required to charge as quickly as possible, so that it is operational when it is needed, so special charging techniques that are also used in electric cars can be used. A further option is for the GUARD to be solar powered for example. This option is currently not very feasible however since solar panels are rather expensive and especially in the Netherlands the sun can appear only sporadically. Furthermore, the GUARD’s main task of guiding and protecting people is most intended for at night time when there is no sun. A combination of the previous two options might however be possible to lengthen the amount of operational hours of the GUARD. The final design choice is to ensure that a decent amount of charging stations are available around the operating space of the GUARD. The solar panels are not yet implemented for the reasons mentioned earlier, but are kept as a backup plan.

Size:

The size of the GUARD must be enough to be noticeable in traffic, but a bigger size will likely also increase costs. A size of 70 - 90 cm (slightly less than half the size of a human) is a good compromise between visibility and costs. From this angle the robot should still be able to observe its surroundings. However, costs can be minimized by simply fitting a larger chassis on a much smaller robot, i.e. most of the space inside the robot is empty. This will make the robot relatively light for its size, so either the weight should be spread out as low as possible or physical constraints should be installed to prevent it from tipping over. The final design choice is to make the robot slightly less than half the size of a human being, as can be seen in the 'Final Design' section. This is a good compromise between visibility and costs, and it ensures that the GUARD looks impressive, but not overly aggressive.

Connectivity:

An important part of the GUARD is connectivity. There are two reasons for this. First off, if a person wants to use a guidance robot, he or she should be able to call the robot and ask for its help. If a robot is nearby, this is no problem, as the robot can understand its client with voice recognition. If a robot is not nearby, however, a client still should be able to quickly ask for help. In order to achieve this there are a couple of options:

•The robot can be connected to a smartphone app, which users can download in order to communicate wirelessly with the robot. The app would only need very basic functions for calling a robot or reserving one in a specific timeslot.

Advantage: the user can simply wait for the robot to arrive. This system is very user friendly. Disadvantage: the robot has to move on its own to the desired location, and while doing so it is not helpful to anyone.

•The robot can be equipped with a GPS-tracker, which again can send its signal to a smartphone app. This would allow users to search for the most nearby robot.

Advantage: the implementation of this idea is slightly easier, as the robot does not need to be able to find its way on its own (This depends of course on the level of the path finding functionality of the robot). Disadvantage: the user has to do more actions, and (s)he has to find the robot himself, under while being unprotected.

The second important connection the robot needs is a connection to the emergency services. This is actually one of the reasons to develop the robot in the first place, so this connection must be strong and stable. This connection should make sure that, if the robot detects an undesirable situation while it is escorting a client, it can immediately make sure that the right and most nearby emergency service is contacted. Furthermore, assuming the robot has GPS connectivity, it can also immediately transfer its location. There are again various ways to establish this connection:

•The robot can be equipped with a SIM card, to allow it to set up a phone connection to an emergency station if needed. This would allow the client to talk, using the robot as the ‘phone’.

Advantage: the client can keep his hands free, and the robot cannot be interrupted as easily as a client using a phone. Furthermore, the robot knows the number and ‘dials’ it in. Disadvantage: if the client cannot speak, the connection is less useful.

•The robot can be programmed to send a standard message, which includes its location, a rough standardized explanation of the situation, and the required form of help.

Advantage: the ‘state’ of the client is less important. Disadvantage: specific situations cannot be explained in detail immediately, but only after the emergency services have arrived.

•The robot can allow an emergency center to look via its built-in camera to a situation (if it has one, of course).

Advantage: part of the decisive process (e.g. is the situation dangerous, which service is needed) is left to humans. Disadvantage: camera’s need to be of good quality and may not be blocked, otherwise the situation cannot be judged.

The final design choice is to make the robot connect to a smartphone app, allowing users to interact with it. It will also be equipped with a touchscreen, to allow users that do not have a smartphone to communicate with it if they need to. Lastly, the robot will have a built-in speaker, allowing its clients to interact with it via voice commands. As for the second part: the robot will be connected to an emergency station, where a human supervisor has the ability to look at the footage from the robot's cameras. The robot can furthermore alert this supervisor in case of really dangerous situations, after which the supervisor can activate the robot's taser if he thinks this is necessary.

Choices:

Whenever a person uses the GUARD, it is possible to set up all sorts of preferences for the robot pathfinding. These choices include the basic settings that all navigation devices can use, such as fastest route, shortest route, etc. But now there’s also a choice to pick a route with the least amount of danger. The settings alone might result in undesirable routes, such as taking too long to reach a destination when set to avoid danger. This is why the user can change the settings to take into account the maximum amount of time the user wants to spend on detours, or override certain streets that have been deemed too dangerous.

Appearance:

Color: For the outward appearance of the robot a color had to be chosen. The choice is police colors, blue with orange stripes. This choice was built on several reasons. It is a color scheme which is a sign of authority and trust, as the people see the police as the authority on security. The GUARD having the same color means that authority of security extends to the robot. The police will also be working with the GUARD thus it is also logical that they all the GUARD has the same color as the equipment of the police. As the police can then easily recognize the robot. Having the police colors also means that potential criminals know that the police will be notified when they do something when a GUARD is near. This will thus increase the deterrence effect which the GUARD will have on crime. This color scheme is also well visible because of its bright colors. This is especially important as the GUARD will primarily work during the dark hours.

Non-Lethal intervention:

For our design we also want in addition to people having a deterrence factor a real physical intervention in case of an undesirable circumstance like physical assault. This situation is of course highly undesirable but also has a low probability of occurring. The feeling of safety, however, is almost as important as actually being safe. As our physical intervention we have chosen to use a Taser as it is a non-lethal intervention with a low probability of permanent damage. Mounting a Taser to our design, however, could cause many legal issues which will have to be explored further. In order to prevent the GUARD from going out of control with a Taser we want to have the Taser only be manually operated by a real human sitting in a control center. From this center the person can view the live camera data of the GUARDs and be notified when a GUARD detects danger. From this data the person will then have to judge whether or not to use the Taser and who to fire it on. This will only be done as a last resort.

Ownership:

Ideally we would like the GUARD to be the property of the local authority of the area where the GUARD is deployed. This also ideally means that the local authorities will pay for the GUARD and allow their citizens to use the GUARD without charging any costs. This is because there should be no price on safety in a public area. In return the local authorities will have a safer city which is always desirable. Also the reputation of the city could be positively influenced by having the GUARD. The current costs of the GUARD are not the focus of this project as they are highly depended on the price of the technology at the time they are made.

Prototype Design

Prototype Design of the G.U.A.R.D.

The final state of the Prototype Design

In the picture 'Prototype Design of the GUARD' to the left, the prototype of the GUARD, that was designed during the middle phase of the project, can be seen. It consists of a robot having about half the height of a human being. To move, the robot makes use of tracks, which ensure high enough speed, and the capability to handle uneven terrain. The tracks are connected to a frame, which is connected to the main structure of the robot with two hinges. The goal of this is that it is easy to implement a mechanism that moves the tracks up and down slightly, to allow the robot to climb stairs. The next step is to implement all the extra features on this structure. An example of this is the camera, which has already been made, and which is lying next to the robot. The robot also already features red/white warning signs on both its front and back, which show / enforce its functionality.

Points of improvement

After this design was discussed within our team, but also with several other people outside of the team, it became clear that there were still some problems, or things to improve, on this prototype. First off, the appearance of the robot was a little too violent, which could not be compensated by the friendly bright green color. This color had to be changed as well, as this was just a prototype color chosen without real supporting reasons. As the GUARD has a serious task, it should be given colors that reflect that. The size of the prototype added to the aggressive appearance, as well as being a problem in daily use, as a GUARD of this size would be hard to fit in certain rooms or spaces (think of an elevator at a train station for example). Therefore, this prototype was not developed to the full final stage. The next step in the design is to take this prototype, and use it as an input to the next design phase, which will lead to the final design.

Final Design

Final Design of the G.U.A.R.D.

Front View of the G.U.A.R.D.

Side View of the G.U.A.R.D.

Design Planning

The plan for the final design is to take the prototype design as an input. The good things of this design will be kept, and the points of improvement will be implemented as much as possible. The points that are kept from the prototype design are: the shape of the body, the tracks and the idea of the moveable tracks, to improve obstacle handling. The main problems that have to be improved are: the violent appearance, the size and the color of the design. The auxiliary accessories have to be designed, other than the camera, which was already finished with the prototype design, and put onto the GUARD as well.

The Final Design

The final design contains the same body shape as the prototype. However, the tracks have been redesigned. To make them look a bit more friendly, the shape has been made triangular, which looks less 'tank-like' and has as an added benefit that triangular constructions are strong. The tracks and the body are not fully interconnected, to allow for some moving of the tracks with respect to the body, improving obstacle handling, and even allowing the climbing of (small) stairs. The overall size of the GUARD was reduced a bit compared to the prototype, which was also done to make it look slightly less aggressive. The colors of the GUARD are dark blue and orange, because these colors are highly visible and give the design some authority. There are five auxiliaries mounted on the final design. The first one is a 360 degree camera, which is mounted on the top. A second camera is mounted to the side of the design, pointing to the front. These cameras work together to enable path finding and facial recognition and ensure good visual capabilities in dangerous situations. The front of the robot contains a touch screen, via which the user can communicate his preferred settings to the GUARD, such as path finding preferences. A microphone is mounted below the touchscreen, to allow the user to give (simple) voice commands to the robot. An LED, which can blink to show the user that he is being listed to, is included. Finally, a taser is mounted to the side, which can be used in highly dangerous situations under human supervision. This concludes the final design. It is shown in the pictures to the left and right of this text. Note that this is the final design of this project, but that there is always room for further (small or large) improvements. This is also shown by the user feedback we got, which can be read in the section 'USE aspects'.

Legal Issues

By deploying a robot into public terrain legality is a big issue. This new technology shouldn't break current laws as that would prevent people from using it. Thus looking at laws which are of interest is important. In this part we focus on Dutch and European laws.

The first legal issue for the GUARD is privacy. As it has cameras knowing what may be done is of great use in finalizing the design of the robot. Responsibility is also a legal issues. As is where the autonomous robot may drive. How its appearance can be within legal standards. And other legalities which come up in human-robot interactions.

Legal issues summary

There are several laws which have influence on the design of the GUARD. As laws do change over time so some more radical designs can be made. Though it is easier to realize the design if the least laws are unnecessarily broken.

To summarize the laws in simple terms: one cannot store personal data in a separate database. These include recognizable voice recordings, facial pictures and daily routine. It is also forbidden to use hidden cameras outside an investigation. And no pictures may be put on a public attainable database. Though monitoring, thus no recording, is in most cases alright.

This has influence on our GUARD in the following ways. The cameras the GUARD use should all be visible as camera or be warned about on the robot. It should not store pictures of all people it passes unless they consented or without reason, like a customer. If the GUARD sees someone committing violence then the GUARD should store his/her picture as it can be used for investigation and that is allowed through the Data Protection Police Files Act.

The most radical thing in terms of privacy is seeing of someone has an open criminal record to look at real time safety routes. And the interconnection of all the GUARD units, so they can share information and act accordingly.

These are two things our group will not concede on in terms of legal problems. This means that first more lenient laws have to be implemented in terms of personal data of criminal records and the sharing of real time camera footage between different systems.

These are non-conceding points because of the central idea of the GUARD, safety. To ensure more real time safety the GUARD needs to know if a threat is nearby. The best way to do this is to know if anyone nearby has done a crime before as those people are most likely to commit another crime. And the GUARDS could interact with each other and notify where those people are then better and safer paths can be chosen. Ensuring better safety of the clients.

Also sharing data means that regular clients are quickly recognized and thus GUARD units can remember the route they last used and can reduce setup time. The legal issues concerning the storing of data in certain databases won’t be tackled as it isn’t of concern in the research that is being done. As the research focusses on safety on the streets and the databases are in the field of cyber security.

Privacy laws:

WBP (Wet Bescherming Persoonsgegevens) (from EU law Data Protection Directive)

Protects personal data from being stored in databases. (though not specifically over camera surveillance.)

Article 10 of the Dutch Constitution

Protection of private life. (Only truly applicable to government – citizen relationship. Other parties may violate this with less repercussions) (but Article 120 of Constitution says that judges may not test if new laws/treaties are following the Constitution)

Article 8 of the European Convention for the Protection of Human Rights and Fundamental Freedoms.

More important than Dutch constitution in law on privacy. May only be circumvented by a public authority in accordance with the law for specific purposes in a democratic society.

Article 17 of the International Covenant on Civil and Political Rights

Protection against arbitrary or unlawful interference with privacy. Everyone has the right to be protected against such influences/attacks.

Article 441b of the Dutch Penal Code (Wetboek van Strafrecht)

Unknown use of technical equipment in a public place is punishable (since 2004).

Article 139f of the Dutch Penal Code

Prohibited to record pictures with a surveillance system in a private place, using a trick. To be punishable pictures have to be made secretly.

Exceptions on Articles 441b and 139f

441b:

Police may use secret cameras in public places if pictures are primarily used for investigation. This is the Data Protection Police Files Act (Wet Politieregister)

139f:

Cameras may be used at workplace if employer suspects illegal behaviour and informs employees of the existence of hidden cameras.

Private Security Organisations and Detective Agencies Act (Wet particuliere beveiligingsorganisaties en recherchebureaus)

Private companies are forbidden to commit security activities without a permit. Private companies may do this to protect people or goods or to prevent disorder.

Works Council Act (Wet op ondernemingsraden)

A Works council has the right to approve an employer to use an employee monitoring system (personeelsvolgsysteem) If no council exist the employer has to inform each individual employee. All camera surveillance systems are (personeelsvolgsystemen). If cameras are used only occasionally no approval from Works Council needed.

Article 21 of the Dutch Copyright Law

May not post pictures/portraits of someone if the person (or relatives if death) has reasonable interest in opposing. (also known as Portrait rights)

Directive 95/46/CE of the European Commission

principles that have to be taken into account for the treatment of personal data and the protection of privacy

Notice: data subjects should be given notice of data collection

Purpose: data should be used for the purposes stated and not for other purposes

Consent: data should not be disclosed without the data subject’s consent

Security: data collected should be kept secure from potential abuses

Disclosure: data subjects should be informed about who is collecting their data and for whom

Accountability: data subjects should have the possibility to hold data collectors accountable for following the above principles

In this Directive is also determined what personal data is in context of surveillance. Speech and images can be personal data if the person can be recognized through this and the objective of the information is surveillance. Location data used for surveillance can also be seen as personal data.

Though article 13.1 of the directive says that member states of the European Union may restrict some of the obligations. This may only be done though if it’s for the investigation of criminal activities. This is only for open public spaces.

For private spaces article 2.e of the directive holds. This says that surveillance in private areas must require the initiative of the private space owner.

In this directive there is also talk about camera deployment by robots in public places by the government. To deploy cameras by robots in public places the following legal premises should be achieved:

- Competence to decide the installation of video surveillance engines is reserved to public security forces (verified by official Commissions). Its use is included in their activities to prevent crime and protect persons and their properties.

- The processing of images or voice is lawful when the authority allows the installation and checks the lawfulness of cameras. The general duty of consent disappears, because there is a legal permission of processing.

- Rights (to access or cancel data) could be denied to benefit general security.

- Databases storing images or sounds depend on public authorities, who obtained the authorization to install cameras. They must notify and register the database and fulfil the obligations of security suitable to the kind of data stored.

Case Laws

Not only articles in the Penal Law code or Constitution or any other law book are important. As we have a judiciary power past lawsuits have influence if the robot will be accepted by the judges or not. These influence how far the laws are implemented and what the correct balance is between security and privacy.

Here are several case laws which influence the privacy laws:

Extent of transparency

The mere fact if a recording is in a public place isn’t enough to reject the right of privacy (1991, Supreme Court)

Kind and extent of intimacy

Mere recording of conservation over phone on tape without the knowledge of the listened to, isn’t enough to interfere with right to privacy, other factors are of importance.(1987, Supreme Court)

Freedom to be yourself

No watching people change clothes, going to the toilet etc. (1994, Court of Justice Amsterdam) (Lüdi case)

Use of pictures

Monitoring of people is okay. If one is recording and storing this data it can be seen an infringement on privacy. If one is distributing the recordings one is quickly punishable for infringement on privacy. Exceptions exist. (Supreme Court)

Openness in video surveillance

If people are monitored without consent an appeal to right of privacy is more easily accepted (1987 Supreme Court)

Systematic use of cameras

Duration of camera surveillance on a person is influential on final verdict if an appeal to right to privacy is accepted.

Simulation

In the simulation the macro level pathfinding of the GUARD will be shown. This macro level pathfinding consists of finding an appropriate route given certain crime rates on a street level. In our model a map of the inner city of Eindhoven is used and various severities of crime rates are displayed in different colors. These crime rates are based on an educated guess by looking at the types of streets and types of shops. The reason why the Eindhoven city centre was chosen as an example is because it is familiar and because quite a lot of data was available about the crime rates. On the website "Eindhoven buurtmonitor" [11] the Eindhoven police provided data of reported crime rates in Eindhoven. All the crime rates are unfortunately given not on a street level but on a neighbourhood level. The inner city of Eindhoven proved to have the most crime per citizens.

The Police reports of Eindhoven inner city in 2015 were:

Numbers are relative occurrences per 1000 citizens.

Pickpocketing: 34.4

Immoral offence: 0.4

Threats: 3.8

Public violence: 1.9

Physical abuse: 14.5

Street-robbery: 1.6

Vandalism: 7.7

Nuisance of beer and drug users: 5.4

Nuisance of youngsters: 1.8

Other nuisances: 10.9

In a survey (on the same website) also 40% of citizens have mentioned that they feel unsafe at times.

Decisions in the simulation

Quite a few decisions had to be made for the end result of our robot. The design process started with choosing what we would like our end result to be. The options were quickly limited to:

Adapting a soccer robot to fit our idea and placing it in a self-created city-like environment. Creating a pathfinding AI and place it in a digital city map.

We chose to create a digital simulation since our interest was more situated in programming rather than building and creating a physical challenge. Also, our design would be less limited in a digital environment. The next decision we had to make was how to improve and expand our design from travel directions to a decision making AI. This was done by adding information about the different sections of the map. For example, areas we thought could be unsafe (pubs/alleyways) were marked on the map. This way the settings of the agent can be altered by users to fulfil their preferences of the route. Users that care more about getting somewhere as fast as possible, only needing the watchful eye and the feeling of protection of the robot, can state they do not care about avoiding potentially dangerous paths. Users that want to optimize the route to be as safe as possible can state they want to avoid these areas at all costs.

Setting one's preferences beforehand can provide a safer feeling than letting users choose a route on the go. For example, when arriving at a crossroads, the agent could let the user choose between routes, saying one is slower but the other is 50% more dangerous. This might cause unease for the user, who has just been informed that the path they would normally take is more susceptible to criminal activities. Our decision therefore was to not inform users about potential crime, unless the route they want to take is vastly more dangerous than another.

A final decision we made is to allow the robot to recognise faces and look them up in criminal data banks. At the moment, this is illegal and also might seem very unethical. Still, we toyed with the idea of optimizing safety more by trying to prevent undesirable confrontations before they appear to form. In our simulation, the GUARD system can spot people with a criminal record and report their position to other GUARD robots. When a potential criminal has been spotted, the robot avoids their presence at all costs.

Code explanation

The simulation is done with netlogo which allows importing map data and provides an easy way to code and display the pathfinding algorithm. The pathfinding used in the simulation is the A* pathfinding where the choice of which node to expand depends on the distance travelled and the remaining distance to the destination. The A* algorithm in the code is largely based on the public model of M. Singh [12]. This code is adapted to fit the needs of our program. On the imported map different colour areas have been added where the green colour represents a slight crime rate, the red colour represents a substantial crime rate and the yellow colour represents a substantial crime rate with in addition many different pubs in the neighbourhood which assumes there are more drunk people in the around. The crime rate on the map might not be fully accurate due to no real data from the police being available. For the different crime rates (different colours) different penalty times have been assigned which are then computed to choose the optimal route. The model works by selecting a starting location (needed for the model but will be done GPS location for the actual GUARD) and a destination. Based on this information the model will choose the optimal route with the standard settings. Optionally the user can select advanced options where he/she can specify the time he/she is willing the spend to avoid crime. Also an extra option whether to avoid pubs, which are prone to drunk people, is part of the advanced option and finally the user can specify a street he/she does not want to visit at all cost. The option to avoid pubs does not guarantee no pubs will be visited but adds an extra penalty to them. With the potential face/ danger recognition the GUARD can also recognise criminals and danger. In order to simulate this behaviour in the model a criminal can be selected and placed which adds a huge penalty time in a radius around the dangerous situation (will be automatic in the real guard). The penalty times are currently based on empirical data of what seems like an acceptable route based on crime rates and travel time. but might be subject to change in the final guard.

Simulation example

To demonstrate the simulation, Three different situations are shown above. In each situation, a map of the inner city of Eindhoven is used. The starting position is always near the center of the map, while the destination is on the northern side.

In the first situation, the agent is set to ignore the difference between safe paths and unsafe paths. This way it just chooses the shortest route towards the destination.

In the second situation, the agent uses the default setup, which is to avoid the unsafe paths to a certain extent. If using a safer route does not save a lot of time by taking it, the faster (and more unsafe) route will be taken anyway. The maximum amount of time spent walking safer routes can be adjusted at will.

In the final situation, a criminal is placed on the most efficient path. The agent tries to avoid criminals at all costs. With the current settings, this means the agent uses an unsafe (red) route to go around the criminal, since this is still generally safer.

Planning

In the picture below, the Gantt chart that was developed at the start of the project is shown. During the project it was used as a guideline, in order to ensure that all deliverables would be finished after 8 weeks.

Sources

[1]: Vreede, J. D. (2016, May 20). AD Misdaadmeter 2015: de ranglijst en alle cijfers. Retrieved March 02, 2017, from http://www.ad.nl/binnenland/ad-misdaadmeter-2015-de-ranglijst-en-alle-cijfers~a6cb0742/

[2]: Phillips, C. (1998). A REVIEW OF CCTV EVALUATIONS: CRIME REDUCTION EFFECTS AND ATTITUDES TOWARDS ITS USE. Crime Prevention Studies,, 10, 1-29. Retrieved March 02

[3]: Sahin, F., Sridhar, P., Horan, B., Raghavan, V., & Jamshidi, M. (2007, October). System of systems approach to threat detection and integration of heterogeneous independently operable systems. In Systems, Man and Cybernetics, 2007. ISIC. IEEE International Conference on (pp. 1376-1381). IEEE.

[4]: Bermejo Nievas, E., Deniz Suarez, O., Bueno García, G., & Sukthankar, R. (2011). Violence detection in video using computer vision techniques. In Computer Analysis of Images and Patterns (pp. 332-339). Springer Berlin/Heidelberg.

[5]: Wu, X., & Zhang, X. (2016). Automated Inference on Criminality using Face Images. arXiv preprint arXiv:1611.04135.

[6]: Revell, T. (2016, 1 December). Concerns as face recognition tech used to ‘identify’ criminals. Retrieved from https://www.newscientist.com/article/2114900-concerns-as-face-recognition-tech-used-to-identify-criminals/

[7]: Brunelli, R., & Poggio, T. (1993). Face recognition: Features versus templates. IEEE transactions on pattern analysis and machine intelligence, 15(10), 1042-1052.

[8]: Fukui, K., & Yamaguchi, O. (2005). Face recognition using multi-viewpoint patterns for robot vision. Robotics Research, 192-201.

[9]: Nouwt, S., de Vries, B. R., & Van der Burgt, D. (2005). Camera surveillance and privacy in the Netherlands.

[10]: Kooij, N. S., Schoute, A., & Poel, M. (2003). The development of a vision system for robotic soccer.

[11]: Local authority of Eindhoven. (2016, June 8th). Eindhoven Buurtmonitor. Retrieved from https://eindhoven.buurtmonitor.nl/

[12]: M. Singh. (2014). Astardemo1. Retrieved from http://ccl.northwestern.edu/netlogo/models/community/Astardemo1