org.apache.hudi.integ.testsuite.reader

类 SparkBasedReader

java.lang.Object

org.apache.hudi.integ.testsuite.reader.SparkBasedReader

public class SparkBasedReader
extends Object

Helper class to read avro and/or parquet files and generate a RDD of GenericRecord.

字段概要

字段

限定符和类型	字段和说明
static String	SPARK_AVRO_FORMAT
static String	SPARK_PARQUET_FORMAT

构造器概要

构造器

构造器和说明
SparkBasedReader()

方法概要

限定符和类型	方法和说明
static org.apache.spark.api.java.JavaRDD<org.apache.avro.generic.GenericRecord>	readAvro(org.apache.spark.sql.SparkSession sparkSession, String schemaStr, List<String> listOfPaths, org.apache.hudi.common.util.Option<String> structName, org.apache.hudi.common.util.Option<String> nameSpace)
static org.apache.spark.api.java.JavaRDD<org.apache.avro.generic.GenericRecord>	readOrc(org.apache.spark.sql.SparkSession sparkSession, List<String> listOfPaths, org.apache.hudi.common.util.Option<String> structName, org.apache.hudi.common.util.Option<String> nameSpace)
static org.apache.spark.api.java.JavaRDD<org.apache.avro.generic.GenericRecord>	readParquet(org.apache.spark.sql.SparkSession sparkSession, List<String> listOfPaths, org.apache.hudi.common.util.Option<String> structName, org.apache.hudi.common.util.Option<String> nameSpace)

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

字段详细资料

SPARK_AVRO_FORMAT

public static final String SPARK_AVRO_FORMAT

另请参阅:

常量字段值

SPARK_PARQUET_FORMAT

public static final String SPARK_PARQUET_FORMAT

另请参阅:

常量字段值

构造器详细资料

SparkBasedReader

public SparkBasedReader()

方法详细资料

readAvro

public static org.apache.spark.api.java.JavaRDD<org.apache.avro.generic.GenericRecord> readAvro(org.apache.spark.sql.SparkSession sparkSession,
                                                                                                String schemaStr,
                                                                                                List<String> listOfPaths,
                                                                                                org.apache.hudi.common.util.Option<String> structName,
                                                                                                org.apache.hudi.common.util.Option<String> nameSpace)

readParquet

public static org.apache.spark.api.java.JavaRDD<org.apache.avro.generic.GenericRecord> readParquet(org.apache.spark.sql.SparkSession sparkSession,
                                                                                                   List<String> listOfPaths,
                                                                                                   org.apache.hudi.common.util.Option<String> structName,
                                                                                                   org.apache.hudi.common.util.Option<String> nameSpace)

readOrc

public static org.apache.spark.api.java.JavaRDD<org.apache.avro.generic.GenericRecord> readOrc(org.apache.spark.sql.SparkSession sparkSession,
                                                                                               List<String> listOfPaths,
                                                                                               org.apache.hudi.common.util.Option<String> structName,
                                                                                               org.apache.hudi.common.util.Option<String> nameSpace)