Added TruncatedSVD module #302
Conversation
Co-authored-by: José Valim <[email protected]>
|
|
||
| defnp fit_transform_n(x, opts) do | ||
| module = fit_n(x, opts) | ||
| Nx.dot(x, Nx.transpose(module.components)) |
There was a problem hiding this comment.
Use here Nx.dot/4 which automatically transposes particular tensors, depending on axes you provided
| {q, _a, _a_t, _i, _n_iter} = | ||
| while {q, a, a_t, i = Nx.tensor(1), n_iter}, Nx.less(i, n_iter) do | ||
| {q, _} = Nx.LinAlg.qr(Nx.dot(a, q)) | ||
| {q, _} = Nx.LinAlg.qr(Nx.dot(a_t, q)) | ||
| {q, a, a_t, i + 1, n_iter} | ||
| end |
There was a problem hiding this comment.
| {q, _a, _a_t, _i, _n_iter} = | |
| while {q, a, a_t, i = Nx.tensor(1), n_iter}, Nx.less(i, n_iter) do | |
| {q, _} = Nx.LinAlg.qr(Nx.dot(a, q)) | |
| {q, _} = Nx.LinAlg.qr(Nx.dot(a_t, q)) | |
| {q, a, a_t, i + 1, n_iter} | |
| end | |
| {q, _} = | |
| while {q, {a, a_t, i = Nx.tensor(1), n_iter}}, Nx.less(i, n_iter) do | |
| {q, _} = Nx.LinAlg.qr(Nx.dot(a, q)) | |
| {q, _} = Nx.LinAlg.qr(Nx.dot(a_t, q)) | |
| {q, {a, a_t, i + 1, n_iter}} | |
| end |
| {u, sigma, vt} = randomized_svd(x, opts) | ||
| {_u, vt} = Scholar.Decomposition.PCA.flip_svd(u, vt) | ||
|
|
||
| x_transformed = Nx.dot(x, Nx.transpose(vt)) |
There was a problem hiding this comment.
Same as below, use Nx.dot/4 here
lib/scholar/decomposition/pca.ex
Outdated
| @@ -471,7 +471,8 @@ defmodule Scholar.Decomposition.PCA do | |||
| end | |||
| end | |||
|
|
|||
| defnp flip_svd(u, v) do | |||
| @doc false | |||
| defn flip_svd(u, v) do | |||
There was a problem hiding this comment.
I would pull this out in a separate module in the same folder, e.g. Scholar.Decomposition.Utils or Scholar.Decomposition.Shared. See how it's done in Scholar.Neighbors.Utils.
| ] | ||
|
|
||
| tsvd_schema = [ | ||
| n_components: [ |
There was a problem hiding this comment.
I would rename this to num_components to be consistent with the rest of Scholar.
| type: :pos_integer, | ||
| doc: "Desired dimensionality of output data." | ||
| ], | ||
| n_iter: [ |
| type: :pos_integer, | ||
| doc: "Number of iterations for randomized SVD solver." | ||
| ], | ||
| n_oversamples: [ |
There was a problem hiding this comment.
And same here, num_oversamples.
| type: :pos_integer, | ||
| doc: "Number of oversamples for randomized SVD solver." | ||
| ], | ||
| seed: [ |
There was a problem hiding this comment.
I think you should pass key here, which is of type {:custom, Scholar.Options, :key}.
See how it's done in e.g. Scholar.Clustering.KMeans.
| q_t = Nx.transpose(q) | ||
| b = Nx.dot(q_t, m) |
There was a problem hiding this comment.
Since q_t is being used only for the Nx.dot call below, you can skip calculating it by using Nx.dot/4: b = Nx.dot(q, [-2], m, [-2])
|
💚 💙 💜 💛 ❤️ |
I've added the
TruncatedSVDtransformer. Currently, the only implemented option for the algorithm parameter is randomized. The arpack algorithm is yet to be implemented and it can be added in the future. I believe this module will be a valuable addition, as the PaCMAP algorithm, which is referenced in issue #238, utilizes this method.For reference, here's the Scikit-learn implementation: Scikit-learn Truncated SVD implementation.