Add ground filtering plugin (CSF) to the official repository of cloudcompare

Questions related to plugins development
AndrewRoberts
Posts: 14
Joined: Mon Aug 11, 2014 5:08 pm

Re: Add ground filtering plugin (CSF) to the official repository of cloudcompare

Post by AndrewRoberts »

I am also very interested in this, thanks for sharing! Looking forward to the new beta
daniel
Site Admin
Posts: 7330
Joined: Wed Oct 13, 2010 7:34 am
Location: Grenoble, France
Contact:

Re: Add ground filtering plugin (CSF) to the official repository of cloudcompare

Post by daniel »

Here is the beta version of CC with qCSF (you'll need 7zip to unzip the archive):
http://www.cloudcompare.org/release/Clo ... bin_x64.7z

I haven't made it crash yet. And it's seems pretty efficient to extract DTMs (after a few tests). It's just a bit slow (I wish I could understand all the code comments in Chinese ;).
Daniel, CloudCompare admin
wpqjbzwm
Posts: 37
Joined: Tue May 24, 2016 10:59 am

Re: Add ground filtering plugin (CSF) to the official repository of cloudcompare

Post by wpqjbzwm »

Thanks to all of you who are interested in our algorithm. This is a new filtering method to extract DTM from LiDAR point cloud with only a few easy-to-set parameters. We also have submitted it to a scientific journal. Currently, the revised version of the paper is under review. We will publish our paper here as soon as it is accepted, and more details can be found on this paper. Please cite the paper if you use it for your publication.

In particular, we would like to thanks daniel, who builds a wonderful platform for developers and users. We also appreciate for your great help and quick response.

Actually, this plugin is under development, anyone have any suggestions, please contact us by email : wpqjbzwm@126.com or leave message here.
daniel
Site Admin
Posts: 7330
Joined: Wed Oct 13, 2010 7:34 am
Location: Grenoble, France
Contact:

Re: Add ground filtering plugin (CSF) to the official repository of cloudcompare

Post by daniel »

I cleaned the code, removed unnecessary memory copies as well as unused variables. This reduces the processing time and memory footprint. I also added a lot of safety checks to avoid a crash due to a lack of memory.

I updated the online beta version:
http://www.cloudcompare.org/release/Clo ... bin_x64.7z

I think the biggest optimization that could be done easily would be to remove this heavy mechanism based on a std::map and a key formed with the X and Z coordinates truncated as strings. I'm just not yet confident enough with the code to change this. Can you explain me a bit more what's the exact idea behind this?
Daniel, CloudCompare admin
wpqjbzwm
Posts: 37
Joined: Tue May 24, 2016 10:59 am

Re: Add ground filtering plugin (CSF) to the official repository of cloudcompare

Post by wpqjbzwm »

Hello,
Daniel, thanks for your kind help. Are you learning Chinese? Currenly, all our code comments are in Chinese, perphaps we may translate it to English in the future.
we use std::map because we want to find the nearest LiDAR point for each cloth particle, this really can be optimized. First of all, we project all the LiDAR points and cloth particles to a horizontal plane (which is xz plane, y is the vertical direction). Then for each projected cloth particle, we find the nearest LiDAR point in this plane. Here, we use CGAL to query the N nearest points, and it returns an array with N points. But only the coordinates (x and z) of these points are given, what I need is the ID or index. with ID, I can query the y coordinate of these nearest points. When ID is not available , I just made a std::map which builds a map between xz and y.
So, I think these can be improved. And I also found that the kNN algorithm implemented in CGAL is not very fast. This may be replaced by other well establised kNN library.

In addtion, I have question about cloudcompare. When it loads a very large point cloud file, it may take a relatively long time. But sometimes, I just want to have a quick view of this dataset. Is it possible to read the data every a few points, which makes it faster to read and rotate. This is just like a subsampling of original point cloud. I don't know is it easy to implement ?
wpqjbzwm
Posts: 37
Joined: Tue May 24, 2016 10:59 am

Re: Add ground filtering plugin (CSF) to the official repository of cloudcompare

Post by wpqjbzwm »

Another queation, I find there are several pcl_*.dll files in the this beta version of cloudcompare,
Can pcl functions be used in cloudcompare, such as keypoints and feature descriptors?
Do you have a plan to put more functionalities into PCL wrapper plugin?
Thank you very much!
daniel
Site Admin
Posts: 7330
Joined: Wed Oct 13, 2010 7:34 am
Location: Grenoble, France
Contact:

Re: Add ground filtering plugin (CSF) to the official repository of cloudcompare

Post by daniel »

For the kNN algorithm: CloudCompare has a unique octree structure (CCLib::DgmOctree). And it's very effective ;). I'll try to change that and see if it helps. And more generally for your std::map you should avoid using a string as key (it's very slow to build the two strings from the double coordinates, and it takes a lot of memory). Maybe a simple quantization of the two coordinates would have been sufficient (i.e. knowing the bounding-box you can convert a double coordinate to a 32 or 64 bits integer using the integer's full range).

And while I'm speaking about implementation: one big bottleneck in your code was that you were copying big vectors each time you passed them as parameters to other functions (or as return value). Always prefer to pass them by reference (with '&').

Regarding loading partial clouds from files: it really depends on the file format. Most of the format are read with third party libraries (E57, LAS, PLY, etc.) and therefore it won't be possible to change anything. And for the ASCII files, as you don't know before hand the number of points, it's not easy to skip points. But I guess it should still be possible in a kind of hacky way... We should add this to the TODO list.

And regarding PCL: the PCL dlls are brought by the qPCL plugin. It has been developed by Luca Penasa and I don't know what are his plans for the future. But anyone is free to extend the set of supported functions.

And you can add PCL as a dependency to your own plugin (but be sure that it's really necessary as PCL is huge ;). You can look at how qPCL does it.
Daniel, CloudCompare admin
wpqjbzwm
Posts: 37
Joined: Tue May 24, 2016 10:59 am

Re: Add ground filtering plugin (CSF) to the official repository of cloudcompare

Post by wpqjbzwm »

Thank you very much to point out these problems to improve the efficiency of CSF plugin, we are also seeking to optimize it.
Do you have compared the performance between CCLib::DgmOctree and other well-known kNN library, such as ANN (http://www.cs.umd.edu/~mount/ANN/) and flann (http://www.cs.ubc.ca/research/flann/) ?

In addtion, do you ever have used any other LiDAR filtering method, how do you think of our CSF in aspects of userbility, efficiency and accuracy? This is really important to us. we just want to know some comments about our algorithm.

Thank you again!
daniel
Site Admin
Posts: 7330
Joined: Wed Oct 13, 2010 7:34 am
Location: Grenoble, France
Contact:

Re: Add ground filtering plugin (CSF) to the official repository of cloudcompare

Post by daniel »

I just updated the plugin (same link as before):
http://www.cloudcompare.org/release/Clo ... bin_x64.7z

I used CC's octree instead of CGAL. On the rasterize and distance calculation steps, it goes almost 10 times faster (also because CC's octree is doing things in parallel). CC's octree could be even faster if it was able to work in 2D, but sadly it's 3D only for now.

(You can still revert to the old code by checking the 'QCSF_CGAL_SUPPORT' option in CMake).

I also added an option to export the cloth mesh at the end of the process (I was curious ;):
CSF_cloth.jpg
CSF_cloth.jpg (111.29 KiB) Viewed 209041 times
And regarding your last questions, I compared CC's with 2 other algorithms during my PhD and my Octree was faster. However it mainly depends on what you want to do. CC's octree is very good when applying a process on the whole cloud (which is exactly what we do when computing distances, etc.). But it's not that good for random queries done on a few points only.

I know flann and ANN are very popular. But I feel they are too generic to be very efficient (especially on big point clouds). They are still very good tools when developing code outside of CC.
Daniel, CloudCompare admin
daniel
Site Admin
Posts: 7330
Joined: Wed Oct 13, 2010 7:34 am
Location: Grenoble, France
Contact:

Re: Add ground filtering plugin (CSF) to the official repository of cloudcompare

Post by daniel »

I don't have so much experience in bare-earth extraction plugins (I only used the Rasterize tool and CANUPO to do this so far).

I think your approach is clever and it's very easy to understand how it works (which is important in order to set the parameters correctly and therefore it increases the usability). And for the few clouds I tested it worked pretty well. Especially if I check the 'Slope processing' checkbox (why isn't it checked by default by the way?).

I think other user could tell you more!
Daniel, CloudCompare admin
Post Reply