org.apache.hudi.integ.testsuite.writer

类 SparkAvroDeltaInputWriter

java.lang.Object

org.apache.hudi.integ.testsuite.writer.SparkAvroDeltaInputWriter

所有已实现的接口:

Closeable, AutoCloseable, DeltaInputWriter<org.apache.spark.api.java.JavaRDD<org.apache.avro.generic.GenericRecord>>

public class SparkAvroDeltaInputWriter
extends Object
implements DeltaInputWriter<org.apache.spark.api.java.JavaRDD<org.apache.avro.generic.GenericRecord>>

Spark based avro delta input writer. We don't use this yet since we cannot control result file size.

字段概要

字段

限定符和类型	字段和说明
org.apache.spark.sql.SparkSession	sparkSession

构造器概要

构造器

构造器和说明
SparkAvroDeltaInputWriter(org.apache.spark.sql.SparkSession sparkSession, String schemaStr, String basePath)

方法概要

限定符和类型	方法和说明
boolean	canWrite() Check whether more data can/should be written.
void	close()
DeltaWriteStats	getDeltaWriteStats() Return the statistics of data written.
DeltaInputWriter	getNewWriter() Return a new instance of this writer.
void	writeData(org.apache.spark.api.java.JavaRDD<org.apache.avro.generic.GenericRecord> iData) Generate any type of data.

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

字段详细资料

sparkSession

public org.apache.spark.sql.SparkSession sparkSession

构造器详细资料

SparkAvroDeltaInputWriter

public SparkAvroDeltaInputWriter(org.apache.spark.sql.SparkSession sparkSession,
                                 String schemaStr,
                                 String basePath)

方法详细资料

writeData

public void writeData(org.apache.spark.api.java.JavaRDD<org.apache.avro.generic.GenericRecord> iData)
               throws IOException

从接口复制的说明: DeltaInputWriter

Generate any type of data.

指定者:

writeData 在接口中 DeltaInputWriter<org.apache.spark.api.java.JavaRDD<org.apache.avro.generic.GenericRecord>>

抛出:

IOException

canWrite

public boolean canWrite()

从接口复制的说明: DeltaInputWriter

Check whether more data can/should be written.

指定者:

canWrite 在接口中 DeltaInputWriter<org.apache.spark.api.java.JavaRDD<org.apache.avro.generic.GenericRecord>>

close

public void close()
           throws IOException

指定者:

close 在接口中 Closeable

指定者:

close 在接口中 AutoCloseable

抛出:

IOException

getDeltaWriteStats

public DeltaWriteStats getDeltaWriteStats()

从接口复制的说明: DeltaInputWriter

Return the statistics of data written.

指定者:

getDeltaWriteStats 在接口中 DeltaInputWriter<org.apache.spark.api.java.JavaRDD<org.apache.avro.generic.GenericRecord>>

getNewWriter

public DeltaInputWriter getNewWriter()
                              throws IOException

从接口复制的说明: DeltaInputWriter

Return a new instance of this writer.

指定者:

getNewWriter 在接口中 DeltaInputWriter<org.apache.spark.api.java.JavaRDD<org.apache.avro.generic.GenericRecord>>

返回:

抛出:

IOException