3D Point Cloud Segmentation, Detection & Classification by PointNet in a Low-Cost System

System

Scripts	Datasets
1.retrieveMiData.py	MiData
2.retrieveMiDataCluster.py	MiDataCluster
3.miDataClusterSegmentation.py	ModelNet10
4.pointNet.py
5.segmentationClassification.py

Requirements ‼️

• Python 3.6
• Open3D
• libroyale
• Tensorflow
• Keras
• Trimesh
• Scikit-learn
• Seaborn
• NumPy
• Matplotlib

Usage ⚙️

Acquisition 📸

To capture 3D scenes using pmd Camboard pico flexx (MiData dataset):

python 1.retrieveMiData.py

To capture 3D scenes using pmd Camboard pico flexx filtered by confidence value (MiDataCluster dataset):

python 2.retrieveMiDataCluster.py

Segmentation ✂️

To segment, filter and cluster the scene:

python 3.miDataClusterSegmentation.py

Train 🧠

To train and evaluate a PointNet model:

python 4.pointNet.py

Classification 🗂

To segment and classify objects in a scene:

python 5.segmentationClassification.py

Results 📊

Acquisition

Depth Image

Point cloud

Segmentation

Different type of objects segmented

Train

Confusion matrix

Classification report

Multiple Predictions

Single prediction

Classification

Original point cloud

Clusters in filtered original cloud

Clusters in filtered point cloud

Cluster obtained after segmentation

Sampled cluster

Prediction

Bibliography 📖

Original PointNet implementation: https://github.com/charlesq34/pointnet
Original Keras implementation: https://github.com/keras-team/keras-io/blob/master/examples/vision/pointnet.py