Embedded Motion Control 2018 Group 1

Group Members

TU/e Number	Name	E-mail
1279602	Ahmed Hizar (A.H.) Ahmed Ajmal	a.h.ahmed.ajmal@student.tue.nl
0914013	Jari (J.J.) van Steen	j.j.v.steen@student.tue.nl
0924842	Nazar (N.) Rozkvas	n.rozkvas@student.tue.nl
1279491	Arpit (A.) Aggarwal	a.aggarwal@student.tue.nl
1031018	Willem (W.) Verhoeven	w.b.verhoeven@student.tue.nl

Escape room

This part describes all of our progress in the first challenge of the course, which is the Escape Room challenge.

Initial design

Before the software could be created that makes an escape possible, an initial design report was created to assist in the design of the software. This initial design report can be found here: Initial design report group 1. The most important points from this report will be given on the wiki page.

Requirements & Specifications

The first step in the design process is the creation of requirements for the software, which gives a way to express PICO's required behaviour in the Escape room challenge. To translate this into software, the requirements have to be specified, which is done via specifications. These specifications are used in the software design. Both the requirements and corresponding specifications are given here:

Requirements	Specifications
PICO has to avoid all walls at any time.	PICO should always stay always stay away from walls at least 15 centimeters.
	PICO should aim for an equal distance to the right and left when moving through a corridor between 0.5 and 1.5 meters.
PICO should complete its task well independent of the initial conditions.	PICO has to scan its surroundings with a frequency of at least 5 Hz while moving to prevent bumping into a wall.
PICO should operate fully autonomously.	The final software should not make use of teleoperation.
PICO cannot translate or rotate faster than prescribed.	The maximum translational velocity of PICO is 0.5 m/s and the maximum rotational velocity of PICO is 1.2 rad/s.
PICO has to deal with small imperfections in the real world it operates in.	An uncertainty margin of 0.05 meter should be present when building a world model to deal with imperfections in the real world.
The software should be easy to set up.	The software should be set up according to the information on the wiki page.
PICO has to cross the finish line as fast as possible within the maximum time of 5 minutes.
PICO needs to stop when it crosses the finish line.	PICO needs to stop after it has entered the corridor and there is more than 1 meter distance to the left and to the right
PICO has to identify the exit of the escape room.	An algorithm is created which ensures that PICO identifies an exit between 0.5 and 1.5 meters wide when facing the exit under an angle of 60 degrees or less.
PICO has to keep looking for the exit if it cannot find the exit.	PICO will drive towards the point furthest away from at that moment if the exit is not detected.

Escape strategy

To escape the room as fast as possible for this challenge, a plan has been created on geometric level that makes PICO escape as fast as possible. This plan will be described here by first explaining the plan at the highest level and explaining some of the steps in more detail afterwards.

Overlaying plan

The highest level of this plan will first be described by a simple step-by-step algorithm.

1. Initialize the sensors and actuators.
2. Scan surroundings from the given position and orientation.
3. Try detecting the exit.
4. IF exit is found: Drive towards exit and skip to step 8. ELSE: Turn 180 degrees and perform step 2 and 3 once again.
5. Try detecting the exit.
6. IF exit is found: Drive towards exit and skip to step 8. ELSE: Move into the direction of the furthest point for a quarter of this distance.
7. Return to step 2.
8. Identify the 2 corridor walls and orient the robot to face the exit, then drive forward while remaining in the center.
9. Stop once the finish line has been crossed.

The details of the steps performed in this plan are written below.

Detection plan

Some addition explanation on steps described in the overlaying plan will be described now, starting with the explanation on how to detect the exit and the data in the corridor of the maze.

Line fitting algorithm

As a groundwork of the detection methods, a line fitting algorithm has been created. This is required to localize the exit as well as detecting the walls of the corridor of the maze. This line fit will be based on saved data from the LRF that has been transformed to cartesian coordinates. The line fit algorithm tries to fit a line between 2 different points of the LRF data. To do this, the data between the first and last point is averaged into a smaller number of points. Between the first and last of these averaged points, a line is fitted, which is easy to do for 2 points. By checking that the root mean square of the distances from the other averaged points is smaller than a certain threshold, it is determined whether the initial points form a line. This threshold value is based on the total number of points and the number of averaged points. If this linefit does work, the mathematical formula of the line as y = ax + b can be passed.

Exit detection

A very important task in the overlaying plan is to detect the exit from a certain given orientation. The algorithm on how to do that, using the previously described line fitting algorithm, is described now. The exit detection starts by trying to make a line fit between the first LRF data point and around 30 points away. This number of 30 has been determined by looking at the size of the room and the total amount of LRF data points. If a line can be fit correctly between these points, it means that a wall is detected. If it cannot create a good line fit, no wall is detected.

When a wall has been found, the next step is to extrapolate this line, and check if the distance between this line and the points of the LRF data exceeds a certain threshold at some point for a minimum amount of points in a row. If this is the case, there is either a corner or the exit located here. If the distance from PICO to these points increases, this means that the exit is found, while a corner is found if the distance from PICO to these points decreases. If the exit is located, only one point of the exit is known. Therefore, the line will be extrapolated even more, until the LRF data gives new value that fit on the same line. When this happens, the second exit point has been found, and the location of the exit is fully known. The knowledge of finding a corner is largely ignored for this challenge, the only benefit of knowing the location of the corner is that a new linefit can be made starting at this corner point.

Corridor detection

...

Furthest point detection

...

Driving plan

Once the exit has been detected, a plan for driving towards the target needs to be made, this plan depends on the required task. The different driving tasks will be expanded upon here.

Driving towards exit

In order to drive towards the exit, the coordinate in the center of the two corner points of the corridor is used as a setpoint. PICO will drive towards this setpoint by turning until the exit is faced and after that, continue to drive forward in an open-loop controlled way. However, since errors can be made in this way, this process is continuously executed when finding the exit. This means that a loop will be executed in which the exit is continuously detected, after which PICO faces the exit and drives forward. This process is executed until PICO has reached a certain distance (20 centimeters) of the exit.

Driving through corridor

If the corridor is entered, a different driving plan applies, where the aim is to drive through the center of the wall. In this case, the detection algorithm provides the data of the two lines corresponding to the walls where PICO has to drive through. The next setpoint that PICO drives towards is a small distance (30 centimeters) in front of the current location, and is located in the center of the two walls. Apart from driving towards this point, PICO also aligns its orientation with the two walls. This process is looped until it has been detected that PICO escaped the corridor.

Driving towards furthest point

When no exit can be detected from the current position, PICO has to move towards the furthest point for one quarter of the total distance. This means PICO drives towards the point that is furthest away from the current position based on 2 scans, one from the initial orientation and one from the orientation after a turn of 180 degrees. This point is supplied as a setpoint to which PICO will drive in an open-loop controlled way. Opposite to the other two driving methods, this setpoint is not continuously updated, as the accuracy of moving exactly towards this setpoint is much less important. This means one simple setpoint is supplied, and the task is finished once driving towards this setpoint is finished.

Program structure

The program has been built according to the Initial Design (Initial design report group 1) having the following 3 structural blocks: Detection, Planning and Control. Each of these correspond to a separate class, connected via an additional class 'main'. The functionality of each class is described in more detail below.

State flags

Execution of the program is sequential, such that the methods are called one-by-one depending on the state of the robot. The states are controlled by boolean flags, which include:

• turn - a flag set to "true" by the planning block when no exit is detected and we wish to turn around to examine the other half of the room. It is unchecked when the control block completes the turn;

• drive_frw - a flag set to "true" by the planning block when exit is detected a PICO has to move in that direction or when no exit is detected and the robot should move in the direction of the furthest point detected;

• turned_once - a flag set to true by the planning block after the first turn in the case exit is not detected form the original orientation. This flag ensures that no continuous turning occurs. The value is set to "false" when drive_frw is set to "true";

• in_corridor - a flag set in the main function when the execution cycle for moving towards the exit it completed. This assumes that PICO is at the exit and a different algorithm for setpoints generation is executed in the planning block;

• escaped - a flag set by the detection block, when no reasonable data is detected after passing through the corridor. As this flag is turned to true, the program should stop executing.

Program classes

The whole program consisted of 4 C++ classes:

main.cpp

• This is the main function where the program starts.
• It initializes objects that are required for the hardware components interaction and program execution (e.g. emc::Rate, emc::IO).
• It initializes the objects storing the sensor data, setpoint for the control loop and the furthest detected point. The later one has the same type as setpoints, however initialized separately since the furthest point is updated every iteration of the main while loop.
• Inside the while loop, the three classes are executed and are constantly updated (Detection, Planning, Control).

detection.cpp

• This is the class that reads data from the sensors, filters it and does detection of primitive shapes and their interpretation. Output is wall line equation, corner points of the exit and one point for the furthest detected distance, with corresponding boolean flags
• Detection file includes methods to detect the walls, find exit and furthest point via line-fitting algorithm.
• The line fitting algorithm provides the equation of a line.
• Method to find exit compares the fitted lines with individual measurements, looking for large neighboring differences in the laser data.
• Method to find walls assumes that the data from the front and the sides of the robot should represent lines. This is verified by searching for laser data with distance closer than a specified value. It then fits the line over those points and spits them to the planning block.
• Method for the furthest point searches for the laser scan with the largest distance data.

planning.cpp

• This is the class that interprets the data and sets the setpoints for movement. The output is relative angle to the setpoint and distance to it.
• It fully exploits the flag commands mentioned earlier and bases its decision on them.
• There are two algorithms between which, the distinction is done depending on the phase that the PICO is in.
• Room logic method checks all the data provided by the detection class and interprets it in destination points.
• A series of corridor following commands access the corridor data when the flag in_corridor is checked. It sets a setpoint based on the two parallel lines describing the walls, keeping the robot in the middle of the corridor

driveControl.cpp

• This is the class that converts the setpoints to the commands to the actuators and tracks their execution.
• It heavily relies on the odometry data.
• Based on the flags turn or drive_frw it executes the functions for speed regulation till the odometry data states that PICO is at the desired position.

Simulation

With the program described above, PICO was able to escape the room in the simulation.

Escape room challenge

Unfortunately, the program for the Escape room challenge was not written perfectly and PICO have failed the challenge, not being able to find the exit. As it was found later, the reason for that is the difference execution of the drive controls in the simulator and on the robot. The simulator was built in such a way, that PICO does not stop moving is a given direction till a command is changed. However, the real robot executed the drive commands only for a certain period of time and then it stops.

As a result, during the first operation stage facing the wall, it did not complete the full turn to scan the surroundings and proceeded to the next stage, moving to the furthest point. The furthest point was not selected properly and PICO have hit the wall.

Another conclusion that our group has made is that there should be a direct repulsion algorithm implemented such that in case PICO is located too close to the wall, the first command would be to move from the wall to the safe (predefined) distance and maintain it during the whole challenge. Only if this condition is achieved, the regular program can be run.

Hospital Room

For the final challenge, the task is to explore a Hospital, which consists of corridor and few rooms. PICO has to explore the rooms, return to the initial position, park backwards to the wall and as a second phase of the challenge, find an object that is located somewhere in the Hospital. It is given that the object is placed in a room, for which the most doors have to be entered from the initial position.

Requirements & Specifications

For the hospital room challenge, new requirements and specifications have to be set.

Requirements	Specifications
...	...

Challenge strategy

Overlaying strategy

The Hospital challenge is going to be completed with the following strategy:

Exploration:

1. Move to the end of the corridor, meanwhile counting the number of doors in the corridor.
2. When reached the end of the corridor, enter the closest room.
   1. In the room (not in the entrance of the room), scan the surroundings with the laser.
   2. If 2 corners of the room are identified, turn approximately 180 degrees and scan the environment again. Use few scans to precisely identify the features of the room and their coordinates. Disregard features that are identified only once or twice over all the scans. (When scanning surroundings, PICO should be stationary). When scanning and identifying features for the room corners, also search for the doors.
   3. When the room is scanned, go to the unvisited exit (if such exists). In case there are few exits that are not leading to the hallway, go to the first unvisited room on the right from PICO.
   4. If there is only one exit, go through the exit.
3. Repeat the procedure for each room connected to the corridor. Write the data into the JSON file storing the information about the explored section of the hospital every time PICO return to the hallway.
4. As all the rooms are visited, return to the initial position.
5. Park backwards. Park "blindly", based on the distance to the back wall, obtained before turning around. Also, implement control effort algorithm to prevent PICO from continuously driving into the wall.

Finding object:

1. Compute the destination to the furthest room as a stack of door coordinates that the robot has to enter. Use the known data about the hospital for accurate localization
2. When in the room with the object, search for any unreasonably close laser scan data and consider that as the object.
3. Move to the object and stop near it. Distance to the object is the same as the distance margin to the wall.

Detection/Mapping strategy

<General plan of detection and mapping without diving into the code>

Driving/Planning strategy

<General plan of driving and planning without diving into the code>

Assumptions

It is important to explicitly state the assumptions to the problem to define the working space of the object. From the description of the challenge the following assumptions were made:

• PICO always starts in the hallway
• Rooms might have nested rooms (entrance to a room is via another room)

Next, from the Exploration strategy described in one of the lower sections, the following assumptions are added:

Comparison with the Escape room challenge

The project in general consists of two challenges to test the PICO. Our group considered the challenges as highly interconnected in terms of developments sequence. The Escape room is viewed as a playground for the basic functions that are to be incorporated in the final Hospital Room challenge. It is clear, that in the Hospital challenge, PICO has to perform few tasks as in the Escape room challenge. Namely, the robot will have to identify the room properties (some of them are the same as in the Escape room challenge) and exit or "escape" the room, just as it have done in the first challenge. Therefore, it was decided to reuse as many developments from the previous part, as possible.

In this section, we define what are the changes comparing to the Escape room and what algorithms are reused.

Reused methods from the Escape room challenge

First of all, we maintain the detection algorithms for lines and objects such as exits of the room. The detection methods were proven to work for the previous challenge, however, needed more calibration for certain conditions. We also reuse the objects that are passed around the program that concern detection and setpoint planning. These are structures Exit, Point, Destination. The functions for the planning are also kept, however, the logic is reorganized for clearer code. Finally, the Control module has remained unchanged, since it was proven to work and direct the robot to a defined setpoint expressed as a turn angle and covered distance.

Innovations relative to the Escape room challenge

Comparing to the intermediate challenge (Escape room), there are few innovations that are employed in the program. These include both: new methods required to complete the challenge and new structure of the program execution. The structural changes mostly relate to the execution order and switching phases of the program.

First of all, since PICO has to perform repetitive tasks such as room exploration, passing through the doors in corridor or in the separate rooms, a State machine was added to the program. The State machine has 2-level states: high states for execution of general high tasks such as exploring the hospital, returning to the initial position, etc and low states that concern action planning within rooms or the corridor.

Exploration and object finding strategy.

High states are:

• EXPLORE_HOSPITAL - high state that allows execution of all the low states for exploration phase (identification of the rooms, corridor).
• RETURN_TO_INIT - high state that changes the objective of the first part of the challenge from exploration to returning to the initial position.
• GO_TO_ROOM - the state that symbolizes execution of the second part of the challenge .

Low states are corridor-related and room-related:

• EXPLORE_CORRIDOR - this is the command that is first executed to go to the end of the hallway, counting the number of doors in the corridor. As it is mentioned previously, it is assumed that PICO always starts there. Therefore, this is also the initialized state.
• EXIT_CORRIDOR - state that is on when PICO is in the corridor, but has all the doors in the corridor identified.
• GO_TO_START - state that is active after all the rooms in the corridor (and, respectively, nested rooms) are explored and stored and the robot has to return to the starting point.
• PARKING - state to park backwards at the end of the first part of the challenge

• EXPLORE_ROOM - state that enables execution of the room identification, which includes detection of features and mapping of the environment.
• GO_TO_NEXT_ROOM - state that is activated if there are more than one exit in a room, since according to the challenge strategy, we do not return to the hallway until all the nested rooms are fully explored.
• EXIT - state that commands PICO to exit the room via the only exit is has.
• EXIT_MAIN_ROOM - this is the low state created specifically for exiting to the corridor from the room with more than one door. With this command, no identification or mapping is needed, while the measurement data is used for more accurate localization of the robot.

• STAND_NEXT_TO_OBJECT - the state that commands PICO to go to the object if one is detected in the second part of the challenge.

Program structure

The structure of the program was changed comparing to the escape room challenge. As it is shown on the picture on the right, there are few structural important blocks added. These are State machine, Monitoring, Interpretation, World model and JSON file block. Below we explain the role and the importance of each block.

• State machine - the block that regulates the progress of the PICO in the challenge. Based on the states defined earlier, the activity in all other blocks is regulated.
• Monitoring - it is the block that incorporates all the logic within the program. It is operated based on the state that the robot is in and decides for the routine to be called in order to complete the task.
• Interpretation/Mapping. Since the detection is outputting labeled features such as corners or exits, it is required to group those appropriately in order for other block to use the data. The Interpretation block is responsible for creation of the objects and their conversion to the absolute coordinates.
• World model - it represents a large database with the information about the environment. The World model includes information both, about the progress of the challenge (how many rooms have been visited, how many nested rooms there are, location of the PICO in general terms like room, corridor, nested room, current state of the State machine, etc.) and about the features of the place that it is currently in (room or corridor features). All the other blocks are communicating with the World model to retrieve information or to write it. The World model does not include any logic regarding the operations.
• Planning - the block which generates a set point where PICO should drive to according to the STATE it is in and the data from the world model.
• Control - it gives a control input to PICO.
• JSON file - it is chosen to use the JSON file to store the map in. It makes the program variable storage clear by separating the unused or irrelevant information from the main section of the program. It is also may be used to view the progress of the mapping over time.

The rest of the block keep their role in the software architecture from the Escape room challenge but are extended with new functionalities.

Testing against critical situations

While deciding on the strategy to complete the challenge, a brainstorm session was organized to think through practical aspects that might prevent us from completing the challenge. A short overview of these and their solutions are presented below.

Starting position

---

Problem

If PICO is exiting a nested room and comes back to the main room, how to ensure that the it does not explore the room the second time and does not create a new object?

Solution

Among the other features of the room, it should also include a flag whether the room was previously explored or not.

---

Problem

How to ensure that the PICO is driving to the end of the corridor in the very first phase? What is the target point to move to?

Solution

The detection class also should output the point that is straight ahead of PICO.

---

Identification of rooms and corridor

---

Problem

A complication that could have caused a failure in our program is the initial location of the PICO in the Hospital. It is, indeed, given that PICO is located in the corridor at the start of the challenge, however, it is not specified whether the corridor has all the doors along a straight line from the initial position. For example, the robot might start in the appendix of the corridor, that has a only one "door" according to our conventions (corridor is the room that we start in and it ends in the opposite wall from the initial position). Since write-to JSON file only happens after we return to the corridor, how to store the data about the visited rooms efficiently without overwriting it?

Solution

Use a stack feature to store the properties of the "main room" (which is meant to be a corridor) such as number of doors. Visit each nested room and use a separate stack for those.

---

Problem

How to deal with T-junctions in the corridors?

Solution

Same approach as with the previous problem can be applied. The corridor is considered to be a room and the actual rooms become nested rooms in this interpretation.

If by the end of the exploration, PICO has 2 rooms with the same number of doors to be passed to reach that room, then it has to choose one and if the object is not found, a new path is constructed to the correct room.

---

Final challenge

...

Lessons learned

...