PRE2016 3 Groep7

Group 7 Wiki

Welcome to the wiki page of group 7, where we present the development of our project: the intelligent G.U.A.R.D. robot. Over the course of 8 weeks the progress of our group will be shown here, such as a planning, the research topics (e.g. literature studies, designs, simulations) and the results. Feel free to take a look around!

Subject: After the brainstorming session in week 1, we came up with a couple of application fields that we could direct our project towards. These included ‘Smart homes’, ‘Guidance robots’ and ‘Cleaning’. Out of these fields we chose the topic of ‘Guidance robots’. Now it was time to find a specification within this field, and then we thought of the lack of safety on the streets, that is often highlighted in TV-programs such as ‘Opsporing verzocht’. While streets and buildings are full of cameras in the current day and age, the obtained evidence of criminal actions is often lacking, due to e.g. a too low camera resolution, a wrong mounting angle of the camera, and/or the criminals themselves, that often demolish the cameras during the crime. Could this be improved by using robot technology? We think that is possible, and present the G.U.A.R.D.: the Guidance Unit And Route Director. The G.U.A.R.D. is a robot that is mainly intended for escorting people that feel unsafe during nighttime. It can guide people to e.g. their home, but its main function is to protect them, for example by calling for help (to 112/911) or by planning routes that avoid less safe streets in a city. This robot could be designed with varying specifications, tailored to the exact functionality that is desirable in a given application area (e.g. a metropole, a city or a small village).

Objectives: The objective of our project is to improve safety and accessibility of living environment. This will be done by creating a concept for a guidance robot in order to help people to quickly and safely reach their destinations. Individuals or small groups that feel unsafe or require additional support to reach their destination can request the help of such a guidance robot.

In order to complete the tasks the guidance robot needs to be able to:

- move in order to guide people

- interact with people

- pathfind through real life real complex and unpredictable paths.

- find optimal routes for clients to guide them to their destination quickly and unharmed.

- call for support when a dangerous situation occurs.

Optimal design choices will need to be made for each of these points.

Approach

Our solution to the problem of unsafe streets and possible dangers at night is the use of a Robotic system that will guide its users to a safe area, for example their homes. This system has been dubbed the GUARD-system. (Guidance Unit And Route Director). This system consists primarily of Robotic ‘bodyguards’. These robots can either wait at places where they are most likely needed, patrol around areas or be called upon by using a smartphone app. It is then possible to state your destination to the robot, which will then accompany you to that destination. To create the feeling of safety, it will have practical options such as a connection to the local police station. To implement the GUARD-System we need a robot with integrated area knowledge and pathfinding, as well as camera’s and motion sensors to avoid it from bumping into obstacles. It also needs to be stable and rigid, so users can support themselves if needed. Also important is an internet/phone connection for calling aid when needed. Some voice recognition is needed to optimize the system, this is useful for receiving directions and recognizing danger, for example. Since streets are less safe at night, the GUARD-system will primarily be used then. At daytime, the robots can either be stored away and charge, or they can be used as city guides in touristic areas.

Design

There are a couple of requirements for the G.U.A.R.D robot that it needs to fulfill in order to properly function while maximizing performance and minimizing costs. The robot needs to be able to traverse the cities it is stationed at, at least as fast as a human can walk around, in any sort of weather. It needs to be able overcome all sorts of inclines, but we’ll go for an upper limit of about 30 degrees. This is much more than will be necessary for the cities in the Netherlands, but we would like to future proof our design for other cities with much steeper streets such as San Francisco for example.

The robot also needs to be fairly sturdy and durable, as it will need to run nearly 24/7 in the city while enduring possible abuse from the numerous user in the course of the day. A possibility also exists to implement a capability to support incapacitated people (i.e. drunk people) while escorting them to their destination. It also needs a camera that can view in all directions.

There several different methods of robot locomotion. We feel that due to the cost effectiveness requirement, multiple legs or having snake-like slithering movements should not be used. A drone can be relatively cost effective because it can be used without making any significant alterations to the design and because it can be used to circumvent the problems with the environment. However a wheeled robot is very effective energy wise and will be mainly used in city centers, which are often very wheel friendly. Raising and lowering sets of wheels can be used to get on sidewalks and can keep the robot straight should it come across an incline.

Where the drone fails is in its sturdiness: Heavy winds might cause it to crash to the ground and people might swat it out of the air. Even though the initial investment might be cheaper, repeated replacements might bring that number up to and beyond the price of the other proposed robots. There’s also the problem of battery life. Current drones can’t even fly for more than half an hour before requiring a recharge. Regular robots with wheels are much more efficient in that aspect.

The design also needs to be user friendly in the sense that it shouldn’t be able to hurt the user, even on accident. No sharp edges should be used in the creation of the outer chassis. To prevent it from toppling over, the center of mass should be suitably low to the ground. The part of the chassis closest to the ground might benefit from a sturdy frame that touches the ground and keeps it from turning over should it be toppled too much towards one side.

Tourists should also be able to interact with the robot without using a smartphone app. This can be done using mostly voice commands but a manual input mechanism might be helpful as a backup as it is possible not all languages are correctly implemented or in case something goes wrong with the voice commands.

State of the art

Danger recognition

One of the important technologies on our robot is being able to recognize threats to the escorted client. One main technology which has this are the so-called “smart” CCTVs.

This technology is not yet widely spread. Smart CCTV refers to visual surveillance systems that analyse and interpret video footage by using pattern recognition technologies. This is even so far developed some cameras can notice violence. The CCTV can recognize this through a person’s irrational or random movement usually seen through violence. This is important for GUARD, the escort robot, as it needs to independently notice threats and then notify local authorities.

A technology can also be used to incorperate existing CCTV-networks into our design. This is called a System of Systems. System of systems (SoS) is a network of complex autonomous systems which interchange information. Several aspects are important to make the SoS work.

-Interoperability

The agents in the system should work separately and autonomous. Each system in the SoS should be able to distribute information to all other systems and all systems must be able to interact autonomously inside and outside the system.

-Integration

Each agent should be able to control the SoS and communicate not depending on their characteristics, like software or position. This is the reason why in a SoS all systems need to talk in the same language. Else two agents might not be able to interact directly.

-Autonomy

As said before each agent should act independently and be able to fulfil its task needing no assistance, outside the extra information gained through the SoS.

To incorperate this, a central language between the robots and sensor networks needs to be found. A way with which a threat is separated from a non-threat needs to be obtained from another technology, such as the “smart CCTVs”.

One thing GUARD needs to detect is violence. There are some algorithms which do this exactly. These use a database of sounds and videos of fights, gunshots, blood and explosions. As these are all signs of danger. This is a form of action recognition. Sadly violence detection is one of the least studied versions of this. Enrique Bermejo Nievas et al. (2011) propose that using a database will bring great results detecting violence in certain situations. They wanted to detect violence and created a database of 1000 ice hockey videos which included violence and 1000 action movie fights. They found that a descriptor called MoSIFT, can detect violence in these situations with an accuracy of 90% using the databases. This can be used by our robot. First needed is a huge database of videos of street violence, like beatings, and including a database of sounds which could be sounds of threats or gunfire. This than can be used by a descriptor like MoSIFT or one similar to recognize threats and violence during the escort mission of our robot and thus bringing the client to safety and notifying human authoraties.

Security robots

The threat detection is also common in another technology than CCTVs. The security robots. While not extremely developed the security robots show promising ideas which GUARD can use to know when to notify the police and the person it escorts.

Autonomous security systems have great potential. Though there is one main difference between the security robots and GUARD is that regular security robots are mostly deployed when no person may enter and thus can report all noticed intruders. GUARD on the other hand is used on the public streets. It could give a sort of tag to its client but it will still run into many citizens who aren’t a threat and don’t have to be reported. Thus regular heat and movement detection won’t fully cut it for GUARD. A method to further establish how a threat will be defined is needed.

Thus further research would be to also incorporate the danger recognition of the smart CCTVs into the security robots and maybe even combine the different robots and a CCTV system into a SoS. This could greatly improve the efficiency of the GUARD.

A hugely invested in and developed security robot is the Knightscope K5. This is a security robot designed to patrol bigger open areas of companies’ buildings. The company claims this robot can autonomously patrol a certain defined area and can detect anomalies and report these to a central station which is operated by humans. These anomalies are detected using cameras in the infrared range and visual range. It can also pick up wireless signals and thus detect phones and other internet connected devices. Using this it can detect unfamiliar devices which can be a sign of espionage or intruders. The company is now working on gun detection but the company won’t reveal how this will be done. A smaller version, the K3, is a similar model with slightly less features to be used in more enclosed areas.

If the K5 is as good as the Knightscope company claims then this would be a good starting point from which we can base our own design off from. As both GUARD and the K5 are used in open areas, and both need to detect anomalies. Though GUARD needs to better distinguish between threat and friendly as it will be deployed in a larger more public environment.

Autonomously Moving AI

Another important piece of technology the GUARD-robots need is the ability to move autonomously. This technology is a very busy area of innovation, as it is currently a very important area in the use of self-driving cars. The technology described in this paragraph will primarily be based on the current workings of self-driving cars, since most information and innovation resides in this area.

Autonomous moving is needed for the GUARD robots, because the alternative would require remote controlling. At this point one should wonder if robots are needed at all, since then it would be easier to send actual people instead of letting them control the robots from a distance. To create an autonomously moving robot, a few technologies are required. The most important technology is GPS. Without it, the robot is unable to know its location, route and destination. With the GPS-system, the robot only needs to be able to receive inputs from its users to function on a basic level. With these basics, the robot would in theory be able to move the way it is designed to. However, the problem then lies in the fact that roads aren't flat or unoccupied.

Influence of cameras on crime

To know if the GUARD will help, first there has to be looked at the influence on the next closest technology. The one that is used long enough and widely spread enough is CCTV, or else known as camera surveillance. A study of the effects was done in 1999 in the United Kingdom, the record holder of cameras per capita. This study was done by Coretta Phillips.

The author says the main claimed effects of CCTV were:

1. Caught in the act: catching a crime red-handed could increase chance offenders will be caught and persecuted and thus deterred.

2. Framing: CCTV could reduce crime by deterring away potential offenders who don’t want to be caught on camera

3. Nosyparker effect: reduction of crime could happen because people will use the observed area more, and thus human surveillance will increase as well.

4. Effective Deployment: Better knowledge of risk areas is gained and thus security workers can be better deployed, reducing crime.

5. Publicity (general/specific): As an area is known to be observed criminals will less likely go to this area to perpetrate a felony. As it looks like people take crime more seriously.

6. Time for crime: The longer a crime takes the more likely its caught on camera. Short crime might not be as effected as its longer counterparts.

7. Memory Jogging: publicity encourages potential victims to be more conscious and take precautions.

8. Appeal to Cautious: Those who are more security minded use observed areas more than careless people, thus a displacement of victims is caused.

For our moving robot most effects are of interest. Effects 3 and 7 are less important as our robot moves with our client and does not observe only a certain area. And the robot is the precaution that can be taken and thus already fulfils this role. The author than looked at the actual measured effects of CCTVs on crime.

First she looked at property crime in streets. This showed promising results. Areas which had at least two cameras had a noticeable decrease in crime in comparison to unobserved areas. Also no displacement effect was noticed. Overall crime was reduced, This confirms effect 2, cameras do deter potential criminals. At least for property theft, which also includes vandalism and burglaries. The effects faded slightly though for petty thefts and vehicle thefts.

In areas like markets the effect was also looked at. This gave also promising results as seen in table 1. This is the effect of CCTVs on property crime in the town centre of Newcastle. This shows that in busier areas the effects are very big. These big numbers are attributed to effects 1 and 2.

Table 1. Effects on property crime in Newcastle town centre in a 26-month period after installment of the CCTV system.

In another town centre in Airdrie, Scotland, similar effects were noticed. In total crime was reduced by 21% in two years. Crimes like housebreaking, theft from motor vehicles and shoplifting, reduced by 48%. Arson and mischief, like graffiti, reduced by 19%. And again no major displacement of crime was seen. An exception is shoplifting which did slightly increased in Glasgow until CCTV was installed there. Thus areas without CCTVs did not experience as great benefits from observed areas. A big plus is the major increase in reported crimes. There was a 116% increase in detections improving effects 1 and 4, caught in the act and effective deployment. And similar number were noticed across the country.

There were some areas like Doncaster where the influence of CCTVs was only 6%. There were major reductions in criminal damage, robbery and theft-person, but no decrease of shoplifting, burglary, violence or drug offences. In a very few areas no real effect was noticed. These are mostly indoor housing projects where most of the crimes were done by the residents. Though there were several problems with this study. It was done in less than a year after instalment and the effects might not have been established. And also residents who should have monitored the CCTVs were neglecting it. Only 14% of the residents said to daily watch the monitored areas and 33% said they don’t want to report a crime in fear of retaliation. This could have had huge impacts on the effects of CCTVs.

Requirements GUARD

There are a couple of requirements for the G.U.A.R.D robot that it needs to fulfill in order to properly function while maximizing performance and minimizing costs. The robot needs to be able to traverse the cities it is stationed at, at least as fast as a human can walk around, in any sort of weather. It needs to be able overcome all sorts of inclines, but we’ll go for an upper limit of about 30 degrees. This is much more than will be necessary for the cities in the Netherlands, but we would like to future proof our design for other cities with much steeper streets such as San Francisco for example. The robot also needs to be fairly sturdy and durable, as it will need to run nearly 24/7 in the city while enduring possible abuse from the numerous user in the course of the day. A possibility also exists to implement a capability to support incapacitated people (i.e. drunk people) while escorting them to their destination. It also needs a camera that can view in all directions. There several different methods of robot locomotion. We feel that due to the cost effectiveness requirement, multiple legs or having snake-like slithering movements should not be used. A drone can be relatively cost effective because it can be used without making any significant alterations to the design and because it can be used to circumvent the problems with the environment. However a wheeled robot is very effective energy wise and will be mainly used in city centers, which are often very wheel friendly. Raising and lowering sets of wheels can be used to get on sidewalks and can keep the robot straight should it come across an incline. Where the drone fails is in its sturdiness: Heavy winds might cause it to crash to the ground and people might swat it out of the air. Even though the initial investment might be cheaper, repeated replacements might bring that number up to and beyond the price of the other proposed robots. There’s also the problem of battery life. Current drones can’t even fly for more than half an hour before requiring a recharge. Regular robots with wheels are much more efficient in that aspect. The design also needs to be user friendly in the sense that it shouldn’t be able to hurt the user, even on accident. No sharp edges should be used in the creation of the outer chassis. To prevent it from toppling over, the center of mass should be suitably low to the ground. The part of the chassis closest to the ground might benefit from a sturdy frame that touches the ground and keeps it from turning over should it be toppled too much towards one side.

Tourists should also be able to interact with the robot without using a smartphone app. This can be done using mostly voice commands but a manual input mechanism might be helpful as a backup as it is possible not all languages are correctly implemented or in case something goes wrong with the voice commands.

Movement:

There several options for movement of the GUARD robot. The first major distinction is whether the robot should move over ground or fly in the air: Flying: Advantages: Less obstacles to avoid, not in the way of people, bird eye view, no problems with uneven terrain, cheaper. Disadvantage: Law and legislation currently an issue (think of drones in urban areas), might not always be able to make good recordings of offenders because it is looking down on them, is not more prone to crashing with stormy weather for example, can crash on people, needs to be light (low battery life), cannot be used as support.

Ground: Advantages: Sturdy, can work during all weather, can be used as support disadvantage: needs to avoid all obstacles and be able to work with uneven terrain

Sensing: Vision: Vision is essential for the GUARD since it is needed for detecting obstacles and routes, checking for the Multiple different types of ways of vision possible Camera: There are multiple ways of implementing cameras in the GUARD there could be a single camera that relies on the GUARD to turn in order to focus on important events. The disadvantage is of course if events happen before the GUARD has the ability to turn around. Multiple cameras could be used as well so that the robot can record everything. This does of course increase the costs.

Sound:

Being able to perceive sounds is almost a must for the GUARD as well since it will be able to record what is being said around the robot to give more data if something were to happen. It could also be used to communicate with the GUARD by means of voice recognition. Touch: Touch will most likely not have any added value to the GUARD robot since vision should be able to do most of the sensing. It could be used to detect the robot being stuck against objects but as mentioned previously vision should be able to do the same.

Power:

The GUARD needs to be able to bring people to their destination and thus must have a large enough battery life to be able to travel that far. Preferably the GUARD is able to drive around for an even longer time than that so new clients can be helped quickly. This means that battery life will become a balance between utility and costs. Ideally the GUARD will be able to drive for extended periods of time when it is most likely to be used which will likely be between 22.00 and 3.00. For drones this battery life will probably not be achievable since the drones needs to be light. Charging: In order to keep the GUARD operational as long as possible the GUARD needs to be able to charge quickly. One way to charge the GUARD is by having charging stations distributed throughout the area. These charging stations could also be used as central pick-up-points for the GUARD. It would however still be ideal for the GUARD to charge as quickly as possible so special charging techniques that are also used in electric cars could be used. Another option is for the GUARD to be solar powered for example. This option is currently not very feasible however since solar panels are rather expensive and especially in the Netherlands the sun can appear only sporadically. The GUARD’s main task of guiding and protecting people is most intended for at nighttime when there is no sun. A combination of the previous two options might be possible to lengthen the amount operational hours of the GUARD.

Size:

The size of the GUARD must be enough to be noticeable in traffic but a bigger size will likely also increase costs. A size similar to the soccer robots could be used as compromise between visibility and costs. From this angle the robot should still be able to observe its surroundings.

Legal Issues

By deploying a robot into public terrain legality is a big issue. This new technology shouldn't break current laws as that would prevent people from using it. Thus looking at laws which are of interest is important. In this part we focus on Dutch and European laws.

The first legal issue for the GUARD is privacy. As it has cameras knowing what may be done is of great use in finalizing the design of the robot.

Privacy laws:

WBP (Wet Bescherming Persoonsgegevens) (from EU law Data Protection Directive)

Protects personal data from being stored in databases. (though not specifically over camera surveillance.)

Article 10 of the Dutch Constitution

Protection of private life. (Only truly applicable to government – citizen relationship. Other parties may violate this with less repercussions) (but Article 120 of Constitution says that judges may not test if new laws/treaties are following the Constitution)

Article 8 of the European Convention for the Protection of Human Rights and Fundamental Freedoms.

More important than Dutch constitution in law on privacy. May only be circumvented by a public authority in accordance with the law for specific purposes in a democratic society.

Article 17 of the International Covenant on Civil and Political Rights

Protection against arbitrary or unlawful interference with privacy. Everyone has the right to be protected against such influences/attacks.

Article 441b of the Dutch Penal Code (Wetboek van Strafrecht)

Unknown use of technical equipment in a public place is punishable (since 2004).

Article 139f of the Dutch Penal Code

Prohibited to record pictures with a surveillance system in a private place, using a trick. To be punishable pictures have to be made secretly.

Exceptions on Articles 441b and 139f

441b:

Police may use secret cameras in public places if pictures are primarily used for investigation. This is the Data Protection Police Files Act (Wet Politieregister)

139f:

May be used at workplace if employer suspects illegal behaviour and informs employees of the existence of hidden cameras.

Private Security Organisations and Detective Agencies Act (Wet particuliere beveiligingsorganisaties en recherchebureaus)

Private companies are forbidden to commit security activities without a permit. Private companies may do this to protect people or goods or to prevent disorder.

Works Council Act (Wet op ondernemingsraden)

A Works council has the right to approve an employer to use an employee monitoring system (personeelsvolgsysteem) If no council exist the employer has to inform each individual employee. All camera surveillance systems are (personeelsvolgsystemen). If cameras are used only occasionally no approval from Works Council needed.

Article 21 of the Dutch Copyright Law

May not post pictures/portraits of someone if the person (or relatives if death) has reasonable interest in opposing. (also known as Portrait rights)

Case Laws

Not only articles in the Penal Law code or Constitution or any other law book are important. As we have a judiciary power past lawsuites have influence if the robot will be accepted by the judges or not. These influence how far the laws are implented and what the correct balance is between security and privacy.

Here are several case laws which influence the privacy laws:

Extent of transparency

The mere fact if a recording is in a public place isn’t enough to reject the right of privacy (1991, Supreme Court)

Kind and extent of intimacy

Mere recording of conservation over phone on tape without the knowledge of the listened to, isn’t enough to interfere with right to privacy, other factors are of importance.(1987, Supreme Court)

Freedom to be yourself

No watching people change clothes, going to the toilet etc. (1994, Court of Justice Amsterdam) (Lüdi case)

Use of pictures

Monitoring of people is okay. zif one is recording and storing this data it can be seen an infringement on privacy. If one is distributing the recordings one is quickly punishable for infringement on privacy. Exceptions exist. (Supreme Court)

Openness in video surveillance

If people are monitored without consent an appeal to right of privacy is more easily accepted (1987 Supreme Court)

Systematic use of cameras

Duration of camera surveillance on a person is influential on final verdictif an appeal to right to privacy is accepted.