Skip to content

LiDAR Based Perception

Overview

lidar_based_perception_1

  1. The pointcloud from LiDAR is received by LiDAR CenterPoint for pointcloud object detection, obtaining object classification, position, and size information, and outputting DetectedObjects.

  2. Multi Object Tracker receives DetectedObjects and performs object tracking to obtain object classification, position, shape, velocity, and acceleration information. It outputs TrackedObjects.

  3. Map Based Prediction receives TrackedObjects, predicts object trajectories based on map information, and outputs PredictedObjects.

How To Use

Launch Perception

Launch Autoware

LiDAR CenterPoint

lidar_based_perception_2

Detect objects in pointcloud using CenterPoint, you can see the CAR obejct in the blue box.

Input Topics

Topic Type Description
/sensing/lidar/concatenated/pointcloud sensor_msgs/msg/PointCloud2 realtime pointcloud

Output Topics

Topic Type Description
/perception/object_recognition/detection/centerpoint/objects autoware_auto_perception_msgs/msg/DetectedObjects Detected Objects

Multi Object Tracker

lidar_based_perception_3

multi-frame object association using Data Association, use EKF Tracker to obtain object tracking with a unique uuid. The speed is visualized by the red line.

Data Association

associate the data of the multi-frame perception results, convert it into the Min Cost Max Flow problem, and solve it using mussp to obtain the connection between the multi-frame perception results.

EKF Tracker

Choose different models for different classifications to track the perception results and improve tracking stability and accuracy.

Input Topics

Topic Type Description
/perception/object_recognition/detection/centerpoint/objects autoware_auto_perception_msgs::msg::DetectedObjects detected objects

Output Topics

Topic Type Description
/perception/object_recognition/tracking/objects autoware_auto_perception_msgs/msg/TrackedObjects Tracked objects without trajectory

Map Based Prediction

lidar_based_perception_4 Predict the object's future trajectory based on the map and the tracked object to improve the safety and stability of path planning.

Remove Old Data in History

store the time series data of objects, which includes the position, speed, and time information, to predict if the objects need to lane change or not.

Get Current Lanelet Then Update Objects Data

Detect whether if the object need to change lane or to to do left lane change or right lane change.

Generate predicted Trajectories for objects

According to the lane change information and map information, generate the predicted reference trajectories for objects.

Input Topics

Topic Type Description
/perception/object_recognition/tracking/objects autoware_auto_perception_msgs/msg/TrackedObjects Tracked Objects without trajectory
/map/vector_map autoware_auto_mapping_msgs/msg/HADMapBin lanelet2 map

Output Topics

Topic Type Description
/perception/object_recognition/objects autoware_auto_perception_msgs::msg::PredictedObjects objects with predicted trajectories

Reference