Some of the widely used method are based on a similarity graph made with the local structure. For instance LLE uses the relative distances, which is related to similarities. Using similarities allows the use of sparse techniques. Indeed, a lot of points are not similar, and then the similarities matrix is sparse. This also means that a lot of manifold can be reduced with these techniques, but not with Isomap or the other geodesic-based techniques.
It is worth mentioning that I only implemented Laplacian Eigenmaps with a sparse matrix, due to the lack of generalized eigensolver for sparse matrix, but it will be available in a short time, I hope.
Continue reading Dimensionality reduction: similarities graph and its use
I hope to present here some result in February, but I’ll expose what I’ve implemented so far :
- Laplacian Eigenmaps
- Hessian Eigenmaps
- Diffusion Maps (in fact a variation of Laplacian Eigenmaps)
- Curvilinear Component Analysis (the reduction part)
- NonLinear Mapping (Sammon)
- My own technique (reduction, regression and projection)
- PCA (usual reduction, but robust projection with an a priori term)
The results I will show here are mainly reduction comparison between the techniques, knowing that each technique has a specific field of application : LLE is not made to respect the geodesic distances, Isomap, NLM and my technique are.
As I approach the end of my PhD, I will propose my manifold learning code in a scikit (see this page) in a few weeks. For the moment, I don’t know which scikit will be used, but stay put…
The content of the scikit will be :
- Laplacian eigenmaps
- Diffusion maps