There's no easy way to do this in fact. You can indeed use the visibility flags (this is used by the interactive segmentation tool typically). You flags the points that should be removed and then create a new cloud from this 'selection'.
If you really need to remove a single point, it could be better to write your own function. But the challenge will then be to delete all the features attached to this point (color, normal, scalar fields, reference in scan grids, labels, etc.). Not an easy task!
Daniel, CloudCompare admin