org.apache.mahout.math.hadoop.stochasticsvd

Class SSVDHelper

java.lang.Object

org.apache.mahout.math.hadoop.stochasticsvd.SSVDHelper

public final class SSVDHelper
extends Object

set of small file manipulation helpers.

Method Summary

Modifier and Type	Method and Description
static Iterator<Pair<org.apache.hadoop.io.Writable,Vector>>	drmIterator(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path glob, org.apache.hadoop.conf.Configuration conf, Deque<Closeable> closeables)
static DenseMatrix	drmLoadAsDense(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path glob, org.apache.hadoop.conf.Configuration conf) helper capabiltiy to load distributed row matrices into dense matrix (to support tests mainly).
static double[][]	extractRawData(Matrix m) extracts row-wise raw data from a Mahout matrix for 3rd party solvers.
static DenseSymmetricMatrix	loadAndSumUpperTriangularMatricesAsSymmetric(org.apache.hadoop.fs.Path glob, org.apache.hadoop.conf.Configuration conf) Load multiple upper triangular matrices and sum them up.
static Vector	loadAndSumUpVectors(org.apache.hadoop.fs.Path glob, org.apache.hadoop.conf.Configuration conf)
static UpperTriangular	loadUpperTriangularMatrix(org.apache.hadoop.fs.Path glob, org.apache.hadoop.conf.Configuration conf) Load only one upper triangular matrix and issue error if mroe than one is found.
static void	saveVector(Vector v, org.apache.hadoop.fs.Path vectorFilePath, org.apache.hadoop.conf.Configuration conf) save single vector into hdfs file.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

saveVector

public static void saveVector(Vector v,
                              org.apache.hadoop.fs.Path vectorFilePath,
                              org.apache.hadoop.conf.Configuration conf)
                       throws IOException

save single vector into hdfs file.

Parameters:

v - vector to save

Throws:

IOException

drmIterator

public static Iterator<Pair<org.apache.hadoop.io.Writable,Vector>> drmIterator(org.apache.hadoop.fs.FileSystem fs,
                                                                               org.apache.hadoop.fs.Path glob,
                                                                               org.apache.hadoop.conf.Configuration conf,
                                                                               Deque<Closeable> closeables)
                                                                        throws IOException

Throws:

IOException

drmLoadAsDense

public static DenseMatrix drmLoadAsDense(org.apache.hadoop.fs.FileSystem fs,
                                         org.apache.hadoop.fs.Path glob,
                                         org.apache.hadoop.conf.Configuration conf)
                                  throws IOException

helper capabiltiy to load distributed row matrices into dense matrix (to support tests mainly).

Parameters:

fs - filesystem

glob - FS glob

conf - configuration

Returns:

Dense matrix array

Throws:

IOException

loadAndSumUpperTriangularMatricesAsSymmetric

public static DenseSymmetricMatrix loadAndSumUpperTriangularMatricesAsSymmetric(org.apache.hadoop.fs.Path glob,
                                                                                org.apache.hadoop.conf.Configuration conf)
                                                                         throws IOException

Load multiple upper triangular matrices and sum them up.

Returns:

the sum of upper triangular inputs.

Throws:

IOException

loadAndSumUpVectors

public static Vector loadAndSumUpVectors(org.apache.hadoop.fs.Path glob,
                                         org.apache.hadoop.conf.Configuration conf)
                                  throws IOException

Returns:

sum of all vectors in different files specified by glob

Throws:

IOException

loadUpperTriangularMatrix

public static UpperTriangular loadUpperTriangularMatrix(org.apache.hadoop.fs.Path glob,
                                                        org.apache.hadoop.conf.Configuration conf)
                                                 throws IOException

Load only one upper triangular matrix and issue error if mroe than one is found.

Throws:

IOException

extractRawData

public static double[][] extractRawData(Matrix m)

extracts row-wise raw data from a Mahout matrix for 3rd party solvers. Unfortunately values member is 100% encapsulated in DenseMatrix at this point, so we have to resort to abstract element-wise copying.