3D point cloud is a commonly used data in spatial modeling & understanding. Such a data can be captured from laser scanners, stereo camera, or structured-light cameras. Point cloud processing is essential for applications like self-driving cars, infrastructure inspection, construction modeling, etc.. Here we will go through some basic modules of a point cloud processing pipeline, include 1) clustering, 2) normal vector estimation, 3) point cloud descriptor extraction, 4) point cloud registration and 5) triangulation.

Week #1: 3D Points Class and Basics
For this week, we will start with some basis stuff. There is a basic point cloud stub class that we will use to augment the existing Matlab point cloud class and toolbox for point cloud processing. Note that the point cloud capability is in Matlab R2015b and later.

Download the pts3D Matlab class stub and the sample point cloud file here. Perform the following:

1. Flesh out the plot function so it can plot the data.
2. Flesh out the normalizePose function.
3. Flesh out the size function (look into the pointCloud class to see how to obtain the value).
4. Flesh out the removeInds function (again look into the pointCloud class operations to see how to make this easier).

Notes: Regarding the pose normalization function, here is the math description. You should use the pointCloud class member functions as much as possible. Also the function setdiff is your friend.

/*Grab data sets, learn to load point cloud data and visualize it.

A list of point cloud dataset can be found here.

For those feel comfortable with C++ programming, Point Cloud Library (PCL) is a nice alternative option. */

Week #2: Connected Component Clustering

Flesh out the member functions for performing connected component clustering (clusterByProximity and connectivityMatrix). This will involve:

1. Completing the connectivity matrix member function.
2. Parsing the function to find connected components.
3. Giving each component a different color for plotting purposes.
4. Returning the label of each point in a vector list.

Be careful when you try to run this on arbitrary data. Because the connected components Matrix is memory intensive, it can only be done with point clouds that are relatively small (on the order of 20 thousand points).

Notes: Matlab functions that will be useful include: pdist, squareform. You will also need to figure out how to implement the algorithm through judicious use of temporary variables.

Week #3: Local Normal Vector Estimation

If the point cloud data represents an object as scanned from some device, usually the points lie on the surface of the object. So they are really 2D structurally speaking (local to each point), though they are in 3D. Let's write a function to estimate the local normals to the point cloud, plus one more to visualize them. There are actually two ways to perform the local normal estimation. Let's try out both.

1. One is via principal component analysis, otherwise known as PCA.
2. The other is using the singular value decomposition we know and love (by now).

Week #4: Clustering by Normal Structure.

The connected component clustering does not do too well when different objects are actually connected, say because they are touching, or one is resting on the other. Usually, however, the objects might have differing local normals at the interface where they are touching. Let's add normal agreement to the connected component clustering to see how the normals can be used to differentiate parts of the scene.

Perform a region growing clustering strategy using the connected components for the region expansion candidate, and using the angular difference between normals to assess membership to the same cluster.

/*Read the slides on normal vector estimation.

For Matlab folks, an exemplar implementation of normal vector estimation can be found here. Try to implement your own estimator.

For PCL folks, you may refer to the sample code. Try to implement your own estimator. */

Week #5: Triangulation
Read the page of Delaunay triangulation on Wikipedia. For further understanding, refer to the slides from MIT CSAIL.

NEED TO FLESH OUT MORE.

/* For Matlab folks, an exemplar implementation of Delaunay triangulation can be found here.

For PCL folks, there's no sample code you can use this time :( However you may check the C++ implementation of Delaunay triangulation from here. Try to integrate it into your PCL pipeline. */

Week #1: Point Cloud Registration

A commonly used and somewhat simple method for registering two point clouds (that presumably are of the same object or have significant similar structure), is to use what is called Iterative Closest Point (ICP). Read up on ICP, then implement in Matlab.

MORE DESCRIPTION NEEDED HERE.

/* For Matlab folks, an exemplar implementation of ICP can be found here. Try to implement your own version of ICP.

For PCL folks, you may refer to the sample code. Try to implement your own version of ICP. */

Week #X: Point Cloud Segmentation

/* Go through the page on region growing. As you may notice, there is a sample code of region growing segmentation using PCL on that page. Simple copy and run that code won't count.

For Matlab folks, an exemplar implementation of region growing can be found here. Note it's implemented for 2D image, and you need to extend that to 3D point cloud.

For PCL folks, you may refer to the sample code. Try to implement your own version of region growing. */

__** Week #X: Cloud Proximity to Another Cloud

Get set of points that a given point cloud is near to relative to another point cloud.

Notes: Matlab function: pdist2

TO BE WRITTEN.

ECE4580 Learning Modules