3D point cloud is a commonly used data in spatial modeling & understanding. Such a data can be captured from laser scanners, stereo camera, or structured-light cameras. Point cloud processing is essential for applications like self-driving cars, infrastructure inspection, construction modeling, etc.. Here we will go through some basic modules of a point cloud processing pipeline, include 1) clustering, 2) normal vector estimation, 3) point cloud descriptor extraction, 4) point cloud registration and 5) triangulation.

Week #1: 3D Points Class and Basics
For this week, we will start with some basis stuff. There is a basic point cloud stub class that we will use to augment the existing Matlab point cloud class and toolbox for point cloud processing. Note that the point cloud capability is in Matlab R2015b and later.

Download the pts3D Matlab class stub and the sample Matlab file here. The Matlab file has four point matrices in it.

Perform the following:

1. Flesh out the plot function so it can plot the data (plot the different point matrices).
2. Flesh out the normalizePose function (normalize for the sPts0X point matrices).
3. Flesh out the size function (look into the pointCloud class to see how to obtain the value).
4. Flesh out the removeInds function (again look into the pointCloud class operations to see how to make this easier).
5. Flesh out the keepOnlyInds function.

Notes: Regarding the pose normalization function, here is the math description. You should use the pointCloud class member functions as much as possible. Also the function setdiff is your friend.

Explore & Deliverables: Given what you know about the SVD and how it orders the singular values, relate this to the shapes of the different sPts0X variables. Write in English what you think is happened and what role the SVD plays in doing that. Plot the before and after pose normalization for the sPts0X point sets.

When you load the points into a matrix, use the size or numPoints member functions to list the number of points in the point cloud, for each point matrix provided. Show that you can remove points from a point cloud by plotting the original point cloud and a decimated version obtained by removing more than one third of the points using the removeInds function. Display the number of points before and after the removal operation.

/*Grab data sets, learn to load point cloud data and visualize it.

A list of point cloud dataset can be found here.

For those feel comfortable with C++ programming, Point Cloud Library (PCL) is a nice alternative option. */

Week #2: Connected Component Clustering
Flesh out the member functions for performing connected components clustering (clusterByProximity and connectivityMatrix). This will involve:

1. Completing the connectivity matrix member function.
2. Parsing the matrix to find connected components.
3. Giving each component a different color for plotting purposes.
4. Returning the label of each point in a vector list.

Be careful when you try to run this on arbitrary data. Because the connected components matrix is memory intensive, it can only be done with point clouds that are relatively small (on the order of 20 thousand points). For more than that many points, the best option is to utilize Matlab's internal pointCloud member functions to request the local neighborhood set.

Notes: Matlab functions that will be useful include: pdist, squareform. You will also need to figure out how to implement the algorithm through judicious use of temporary variables.

Explore & Deliverables: Apply to the set of points from the Week #1 data file. The data file clearly has distinct connected components. Show that your code does grab them properly.

Week #3: Local Normal Vector Estimation
If the point cloud data represents an object as scanned from some device, usually the points lie on the surface of the object. So they are really 2D structurally speaking (local to each point), though they are in 3D. Let's write a function to estimate the local normals to the point cloud, plus one more to visualize them. There are actually two ways to perform the local normal estimation. Let's try out both.

1. One is via principal component analysis, otherwise known as PCA.
2. The other is using the SVD we know and love (by now).

For the sample point cloud file given, plot the normals. Do not really plot them all, but rather sub-sample the points array and only compute then plot the normals for those points. The Matlab example linked to in the discussion page for this problem (above link for local normal estimation) shows how to perform that.

Week #4: Clustering by Normal Structure.
The connected component clustering does not do too well when different objects are actually connected, say because they are touching, or one is resting on the other. Usually, however, the objects might have differing local normals at the interface where they are touching. Let's add normal agreement to the connected component clustering to see how the normals can be used to differentiate parts of the scene.

Perform a region growing clustering strategy using the connected components for the region expansion candidate, and using the angular difference between normals to assess membership to the same cluster.

/*Read the slides on normal vector estimation.

For Matlab folks, an exemplar implementation of normal vector estimation can be found here. Try to implement your own estimator.

For PCL folks, you may refer to the sample code. Try to implement your own estimator. */

Week #5: Triangulation
Read the page of Delaunay triangulation on Wikipedia. For further understanding, refer to the slides from MIT CSAIL.

NEED TO FLESH OUT MORE.

/* For Matlab folks, an exemplar implementation of Delaunay triangulation can be found here.

For PCL folks, there's no sample code you can use this time :( However you may check the C++ implementation of Delaunay triangulation from here. Try to integrate it into your PCL pipeline. */

Week #1: Point Cloud Registration

A commonly used and somewhat simple method for registering two point clouds (that presumably are of the same object or have significant similar structure), is to use what is called Iterative Closest Point (ICP). Read up on ICP, then implement in Matlab.

MORE DESCRIPTION NEEDED HERE.

/* For Matlab folks, an exemplar implementation of ICP can be found here. Try to implement your own version of ICP.

For PCL folks, you may refer to the sample code. Try to implement your own version of ICP. */

Week #X: Point Cloud Segmentation

/* Go through the page on region growing. As you may notice, there is a sample code of region growing segmentation using PCL on that page. Simple copy and run that code won't count.

For Matlab folks, an exemplar implementation of region growing can be found here. Note it's implemented for 2D image, and you need to extend that to 3D point cloud.

For PCL folks, you may refer to the sample code. Try to implement your own version of region growing. */

__** Week #X: Cloud Proximity to Another Cloud

Get set of points that a given point cloud is near to relative to another point cloud.

Notes: Matlab function: pdist2

TO BE WRITTEN.

ECE4580 Learning Modules