Add ordinations (MDS and PCA)

[kescobo]

EDIT: Wait to merge for a tagged version of MultivariateStats that includes JuliaStats/MultivariateStats.jl#85

Unfortunately, the interface is not super generic across ordinations, but here at least is PCA:

using MultivariateStats, RDatasets, StatsPlots

iris = dataset("datasets", "iris")
Xtr = convert(Matrix, iris[:, 1:4])
M = fit(PCA, Xtr; maxoutdim=2)

plot(M, group=iris.Species)

This and PCoA are the only ones I'm familiar with. I could try some others, but I'm not confident I'd do the correctly.

[kescobo]

Maybe don't merge until we get an answer about the aspect ratio thing. Also might want to regress that previous one, since it might break things if folks don't have MultivariateStats.jl master. I can put the MDS (and any other ordinations you think I should try) here.

[mkborregaard]

Good point - this is awesome!

[mkborregaard]

I've reverted the MDS PR, so if you rebase this and keep that code when resolving the conflict it should work cleanly I guess?

[kescobo]

Link to discourse post about aspect ratio.

[kescobo]

I'm unclear if I should stop using projection for PCA as well.

[kescobo]

According to my labmate Siyuan, the axes of PCA should be scaled also, but I'm running into issues. Something is clearly wrong with using the projection, a quick google search for PCA on iris all have the opposite orientation to my example above (so does the MultivariateStats example:

[mkborregaard]

There's been some discussion on rows vs columns for input data in MultivariateStats - is this related to that?

[kescobo]

It's related, but it's not only that. At the moment, one can get the scaled axes by using transform(pca, matrix), but not from the PCA type alone. At least not as far as I can tell.

[mkborregaard]

I've reached the point where I'm happy to accept anything you find best :-)

[kescobo]

Lol, K. I think it's going to require MultivariateStats exposing eigenvectors of the PCA. Since we need to wait for a release before this gets merged anyway, I'll just wait until then before attempting to fix. Will let you know.

[asinghvi17]

Bump :)

[kescobo]

@asinghvi17 I think you need to bump MultivariateStats - this depends on a PR that's merged in master over there but doesn't have a release yet.

[kescobo]

The change to MDS has finally been merged, so this can move forward. Unfortunately, I'm stuck with PCA. The MDS works great:

using MultivariateStats, RDatasets, StatsPlots

iris = dataset("datasets", "iris")
X = convert(Matrix, iris[:, 1:4])
M = fit(MDS, X'; maxoutdim=2)

plot(M, group=iris.Species)

Unfortunately, PCA isn't working so well:

using MultivariateStats, RDatasets, StatsPlots

iris = dataset("datasets", "iris")
X = convert(Matrix, iris[:, 1:4])
M = fit(PCA, X; maxoutdim=2)

plot(M, group=iris.Species)

I think this is because the eigenvectors (acquired using transform(pca) are supposed to be scaled by the eigenvalues, which I don't think come along for the ride in the PCA object.

I'm wondering if this PR should go back to being just about MDS, and leave PCA for someone a bit more knowledgable about linear algebra...

[kescobo]

I think this is ready to go as well. I removed the PCA recipe, but will refactor once JuliaStats/MultivariateStats.jl#109 is worked out.

[mkborregaard]

Thanks @kescobo . And I see we went with aspect_ratio = 1 in the end 😂

[kescobo]

Yeah, you bullied me into it 😛