Skip to content

Commit

Permalink
minor fixes. (ApolloAuto#4578)
Browse files Browse the repository at this point in the history
* minor fixes.

* minor fixes.

* fix two typos.
  • Loading branch information
csukuangfj authored and Capri2014 committed Jun 21, 2018
1 parent 3fc3c7f commit dc146b6
Show file tree
Hide file tree
Showing 76 changed files with 327 additions and 324 deletions.
24 changes: 12 additions & 12 deletions docs/specs/3d_obstacle_perception.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ The table below describes the usage of parameters for HDMap ROI Filter.
Convolutional Neural Networks (CNN) Segmentation
------------------------------------------------

After identifing the surrounding environment using the HDMap ROI filter, Apollo obtains the filtered point cloud that includes *only* the points inside the ROI (i.e., the drivable road and junction areas). Most of the background obstacles, such as buildings and trees around the road region, have been removed, and the point cloud inside the ROI is fed into the segmentation module. This process detects and segments out foreground obstacles, e.g., cars, trucks, bicycles, and pedestrians.
After identifying the surrounding environment using the HDMap ROI filter, Apollo obtains the filtered point cloud that includes *only* the points inside the ROI (i.e., the drivable road and junction areas). Most of the background obstacles, such as buildings and trees around the road region, have been removed, and the point cloud inside the ROI is fed into the segmentation module. This process detects and segments out foreground obstacles, e.g., cars, trucks, bicycles, and pedestrians.

| Input | Output |
| ---------------------------------------- | ---------------------------------------- |
Expand Down Expand Up @@ -165,10 +165,10 @@ Given this graph, Apollo adopts a compressed Union Find algorithm to efficiently

<div align=center>Figure 3 Illustration of obstacle clustering</div>

- The red arrow represents the object center offset prediction for each cell.
- The red arrow represents the object center offset prediction for each cell.
- The blue mask corresponds to the object cells for which the objectness probability is no less than 0.5.
- The cells within the solid red polygon compose a candidate object cluster.
- The red filled five-pointed stars indicate the root nodes (cells) of sub-graphs that correspond to the connected components.
- The red filled five-pointed stars indicate the root nodes (cells) of sub-graphs that correspond to the connected components.

One candidate object cluster can be composed of multiple neighboring connected components whose root nodes are adjacent to each other.

Expand All @@ -178,7 +178,7 @@ The class probabilities are summed up over the nodes (cells) within the object c

After clustering, Apollo obtains a set of candidate object clusters each of which includes several cells. In the post-processing step, Apollo first computes the detection confidence score and object height for each candidate cluster by averaging the positiveness and object height values of its involved cells respectively. Then, Apollo removes the points that are too high with respect to the predicted object height and collects the points of valid cells for each candidate cluster. Finally, Apollo removes the candidate clusters that have either a very low confidence score or a small number of points, to output the final obstacle clusters/segments.

Set the user-defined parameters in the configuration file `modules/perception/model/cnn\_segmentation/cnnseg.conf`.
Set the user-defined parameters in the configuration file `modules/perception/model/cnn\_segmentation/cnnseg.conf`.

The table below explains the parameter usage and default values for CNN Segmentation.

Expand Down Expand Up @@ -278,31 +278,31 @@ To smooth the obstacle type and reduce the type switch over the entire trajector

where the unary term acts on each single node, while the binary one acts on each edge.

The probability in the unary term is the class probability output by the CNN-based prediction, and the state transition probability in the binary term is modeled by the obstacle type transition from time *t-1* to time *t*, which is statistically learned from large amounts of obstacle trajectories. Specifically, Apollo also uses a learned confusion matrix to indicate the probability of changing from the predicted type to ground truth type to optimize the original CNN-based class probability.
The probability in the unary term is the class probability output by the CNN-based prediction, and the state transition probability in the binary term is modeled by the obstacle type transition from time *t-1* to time *t*, which is statistically learned from large amounts of obstacle trajectories. Specifically, Apollo also uses a learned confusion matrix to indicate the probability of changing from the predicted type to ground truth type to optimize the original CNN-based class probability.

Using the Viterbi algorithm, the sequential obstacle type is optimized by solving the following problem:
Using the Viterbi algorithm, the sequential obstacle type is optimized by solving the following problem:

![CRF_eq3](https://github.com/ApolloAuto/apollo/blob/master/docs/specs/images/3d_obstacle_perception/CRF_eq3.png)

## Radar Detector

Given the radar data from the sensor, follow a basic process such as the one described below.
Given the radar data from the sensor, follow a basic process such as the one described below.

First, the track ID should be extended, because Apollo needs a global track ID for ID association. The original radar sensor provides an ID with only 8 bits, so it is difficult to determine if two objects with the same ID in two adjacent frames denote a single object in the tracking history, especially if there is a frame dropping problem. Apollo uses the measurement state provided by the radar sensor to handle this problem. Meanwhile, Apollo assigns a new track ID to the object that is far away from the object with the same track ID as in the previous frame.
First, the track ID should be extended, because Apollo needs a global track ID for ID association. The original radar sensor provides an ID with only 8 bits, so it is difficult to determine if two objects with the same ID in two adjacent frames denote a single object in the tracking history, especially if there is a frame dropping problem. Apollo uses the measurement state provided by the radar sensor to handle this problem. Meanwhile, Apollo assigns a new track ID to the object that is far away from the object with the same track ID as in the previous frame.

Second, use a false positive filter to remove noise. Apollo sets some threshold via radar data to filter results that could be noise. Then, objects are built according the radar data as a unified object format. Apollo translates the objects into world coordinates via calibration results. The original radar sensor provides the relative velocity of the object, so Apollo uses the host car velocity from localization. Apollo adds these two velocities to denote the absolute velocity of the detected object.
Second, use a false positive filter to remove noise. Apollo sets some threshold via radar data to filter results that could be noise. Then, objects are built according the radar data as a unified object format. Apollo translates the objects into world coordinates via calibration results. The original radar sensor provides the relative velocity of the object, so Apollo uses the host car velocity from localization. Apollo adds these two velocities to denote the absolute velocity of the detected object.

Finally, the HDMap ROI filter is used to obtain the interested objects. Only objects inside the ROI are used by the sensor fusion algorithm.

## Obstacle Results Fusion

The sensor fusion module is designed to fuse LiDAR tracking results and radar detection results. Apollo first matches the sensor results with the fusion items by tracking their IDs. Then it computes the association matrix for unmatched sensor results and unmatched fusion items to get an optimal matching result.
The sensor fusion module is designed to fuse LiDAR tracking results and radar detection results. Apollo first matches the sensor results with the fusion items by tracking their IDs. Then it computes the association matrix for unmatched sensor results and unmatched fusion items to get an optimal matching result.

For the matched sensor result, update the corresponding fusion item using the Adaptive Kalman Filter. For the unmatched sensor result, create a new fusion item. Remove any stale unmatched fusion items.
For the matched sensor result, update the corresponding fusion item using the Adaptive Kalman Filter. For the unmatched sensor result, create a new fusion item. Remove any stale unmatched fusion items.

### Fusion Items Management

Apollo has the concept of publish-sensor. The given radar results are cached. The given LiDAR results trigger the fusion action. The frequency of sensor fusion output is the same as the frequency of publish sensor. Apollo's publish-sensor is LiDAR. The sensor results feed the fusion pipeline sorted by the sensor time stamp. Apollo keeps all sensor results. The object survival time is set for different sensor objects in Apollo. An object is kept alive if at least one sensor result survives. The Apollo perception module provides fusion results of LiDAR and radar in the short-range area around the car and radar-only results for the long distance.
Apollo has the concept of publish-sensor. The given radar results are cached. The given LiDAR results trigger the fusion action. The frequency of sensor fusion output is the same as the frequency of publish sensor. Apollo's publish-sensor is LiDAR. The sensor results feed the fusion pipeline sorted by the sensor time stamp. Apollo keeps all sensor results. The object survival time is set for different sensor objects in Apollo. An object is kept alive if at least one sensor result survives. The Apollo perception module provides fusion results of LiDAR and radar in the short-range area around the car and radar-only results for the long distance.

### Sensor Results to Fusion Lists Association

Expand Down
16 changes: 8 additions & 8 deletions docs/specs/Apollo_2.0_Software_Architecture.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Apollo 2.0 Software Architecture
# Apollo 2.0 Software Architecture




Core software modules running on the Apollo 2.0 powered autonomous vehicle include:

Expand Down Expand Up @@ -45,17 +45,17 @@ The other localization method is the Multiple Sensor Fusion (MSF) method, where

## Routing

The routing module needs to know the routing start point and routing end point, to compute the passage lanes and roads. Usually the routing start point is the autnomous vehicle location. The important data interface is an event triggered function called `OnRoutingRequest`, in which `RoutingResponse` is computed and published as shown below.
The routing module needs to know the routing start point and routing end point, to compute the passage lanes and roads. Usually the routing start point is the autonomous vehicle location. The important data interface is an event triggered function called `OnRoutingRequest`, in which `RoutingResponse` is computed and published as shown below.

![img](https://github.com/ApolloAuto/apollo/blob/master/docs/specs/images/routing.png)

## Planning

Apollo 2.0 uses several information sources to plan a safe and collision free trajectory, so the planning module interacts with almost every other module,
Apollo 2.0 uses several information sources to plan a safe and collision free trajectory, so the planning module interacts with almost every other module.

Initially, the planning module takes the prediction output. Because the prediction output wraps the original perceived obstacle, the planning module subscribes to the traffic light detection output rather than the perception obstacles output.
Initially, the planning module takes the prediction output. Because the prediction output wraps the original perceived obstacle, the planning module subscribes to the traffic light detection output rather than the perception obstacles output.

Then, the planning module takes the routing output. Under certain scenarios, the planning module might also trigger a new routing computation by sending a routing request if the current route cannot be faithfully followed.
Then, the planning module takes the routing output. Under certain scenarios, the planning module might also trigger a new routing computation by sending a routing request if the current route cannot be faithfully followed.

Finally, the planning module needs to know the location (Localization: where I am) as well as the current autonomous vehicle information (Chassis: what is my status). The planning module is also triggered by a fixed frequency, and the main data interface is the `OnTimer` callback function that invokes the `RunOnce` function.

Expand All @@ -65,7 +65,7 @@ The data dependencies such as chassis, localization, traffic light, and predicti

## Control

As described in the planning module, control takes the planned trajectory as input, and generates the control command to pass to CanBus. It has three main data interfaces: OnPad, OnMonitor, and OnTimer.
As described in the planning module, control takes the planned trajectory as input, and generates the control command to pass to CanBus. It has three main data interfaces: OnPad, OnMonitor, and OnTimer.

![img](https://github.com/ApolloAuto/apollo/blob/master/docs/specs/images/control_1.png)

Expand All @@ -79,7 +79,7 @@ The CanBus has two data interfaces as shown below.

![img](https://github.com/ApolloAuto/apollo/blob/master/docs/specs/images/canbus_1.png)

The first data interface is a timer-based publisher with the callback function `OnTimer`. This data interface periodically publishes the chassis information as well as chassis details, if enabled.
The first data interface is a timer-based publisher with the callback function `OnTimer`. This data interface periodically publishes the chassis information as well as chassis details, if enabled.

![img](https://github.com/ApolloAuto/apollo/blob/master/docs/specs/images/canbus_2.png)

Expand Down
10 changes: 5 additions & 5 deletions docs/specs/perception_apollo_2.5.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Apollo 2.5 *does not* support a high curvature road, a road without lane marks i
- ***Recommended road***
- ***Road with clear white lane lines on both sides***

- ***Avoid***
- ***Avoid***
- ***High curvature road***
- ***Road without lane line marks***
- ***Intersection***
Expand Down Expand Up @@ -44,7 +44,7 @@ Given a 2D box, with its 3D size and orientation in camera, this module searches

#### Object tracking

The object tracking module utilizes multiple cues such as 3D position, 2D image patches, 2D boxes, or deep learning ROI features. The tracking problem is formulated ase multiple hypothesis data association by combining the cues efficiently to provide the most correct association between tracks and detected object, thus obtaining correct ID association for each object.
The object tracking module utilizes multiple cues such as 3D position, 2D image patches, 2D boxes, or deep learning ROI features. The tracking problem is formulated as multiple hypothesis data association by combining the cues efficiently to provide the most correct association between tracks and detected object, thus obtaining correct ID association for each object.

### Lane detection/tracking
Among static objects, we will handle lane lines only in Apollo 2.5. The lane is for both longitudinal and lateral control. A lane itself guides lateral control and an object in the lane guides longitudinal control.
Expand All @@ -54,10 +54,10 @@ Among static objects, we will handle lane lines only in Apollo 2.5. The lane is
The lane can be represented by multiple sets of polylines such as next left lane line, left line, right line, and next right line. Given a heatmap of lane lines from the deep network, the segmented binary image is generated by thresholding. The method first finds the connected components and detects the inner contours. Then it generates lane marker points based on the contour edges in the ground space of ego-vehicle coordinate system. After that, it associates these lane markers into several lane line objects with corresponding relative spatial (e.g., left(L0), right(R0), next left(L1), next right(L2), etc.) labels.

### CIPV (Closest-In Path Vehicle)
A CIPV is a the closest vehicle in our ego-lane. An object is represented by 3D bounding box and its 2D projection from the top-down view localizes the object on the ground. Then, each object will be checked if it is in the ego-lane or not. Among the objects in our ego-lane, the closest one will be selected as a CIPV.
A CIPV is the closest vehicle in our ego-lane. An object is represented by 3D bounding box and its 2D projection from the top-down view localizes the object on the ground. Then, each object will be checked if it is in the ego-lane or not. Among the objects in our ego-lane, the closest one will be selected as a CIPV.

### Radar + camera fusion
Given multiple sensors, their output should be combined in a synergic fashion. Apollo 2.5. introduces a sensor set with a radar and a camera. For this process, both sensors need to be calibrated. Each sensor will be calibrated using the same method introduced in Apollo 2.0. After calibration, the output will be represented in a 3-D world coordinate and each output will be fused by their similarity in location, size, time and the utility of each sensor. After learning the utility function of each sensor, the camera contributes more on lateral distance and the radar contributes more on longitudinal distance measurement.
Given multiple sensors, their output should be combined in a synergic fashion. Apollo 2.5. introduces a sensor set with a radar and a camera. For this process, both sensors need to be calibrated. Each sensor will be calibrated using the same method introduced in Apollo 2.0. After calibration, the output will be represented in a 3-D world coordinate and each output will be fused by their similarity in location, size, time and the utility of each sensor. After learning the utility function of each sensor, the camera contributes more on lateral distance and the radar contributes more on longitudinal distance measurement.

### Virtual lane
All lane detection results will be combined spatially and temporarily to induce the virtual lane which will be fed to planning and control. Some lane lines would be incorrect or missing in a certain frame. To provide the smooth lane line output, the history of lane lines using vehicle odometry is used. As the vehicle moves, the odometer of each frame is saved and lane lines in previous frames will be also saved in the history buffer. The detected lane line which does not match with the history lane lines will be removed and the history output will replace the lane line and be provided to the planning module.
Expand All @@ -75,7 +75,7 @@ The input of PnC will be quite different with that of the previous lidar-based s
- Type: CIPV, PIHP, others
- Classification type: car, truck, bike, pedestrian

The world coordinate will be ego-coordinate in 3D where the rear center axel is an origin.
The world coordinate will be ego-coordinate in 3D where the rear center axle is an origin.

## References
[1] J Redmon, S Divvala, R Girshick, A Farhadi, "You only look once: Unified, real-time object detection," CVPR 2016
Expand Down
Loading

0 comments on commit dc146b6

Please sign in to comment.