org.apache.hudi.internal

类 HoodieDataSourceInternalWriter

java.lang.Object

org.apache.hudi.internal.HoodieDataSourceInternalWriter

所有已实现的接口:

org.apache.spark.sql.sources.v2.writer.DataSourceWriter

public class HoodieDataSourceInternalWriter
extends Object
implements org.apache.spark.sql.sources.v2.writer.DataSourceWriter

Implementation of DataSourceWriter for datasource "hudi.internal" to be used in datasource implementation of bulk insert.

构造器概要

构造器

构造器和说明
HoodieDataSourceInternalWriter(String instantTime, HoodieWriteConfig writeConfig, org.apache.spark.sql.types.StructType structType, org.apache.spark.sql.SparkSession sparkSession, org.apache.hadoop.conf.Configuration configuration, org.apache.spark.sql.sources.v2.DataSourceOptions dataSourceOptions, boolean populateMetaFields, boolean arePartitionRecordsSorted)

方法概要

限定符和类型	方法和说明
void	abort(org.apache.spark.sql.sources.v2.writer.WriterCommitMessage[] messages)
void	commit(org.apache.spark.sql.sources.v2.writer.WriterCommitMessage[] messages)
org.apache.spark.sql.sources.v2.writer.DataWriterFactory<org.apache.spark.sql.catalyst.InternalRow>	createWriterFactory()
void	onDataWriterCommit(org.apache.spark.sql.sources.v2.writer.WriterCommitMessage message)
boolean	useCommitCoordinator()

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

构造器详细资料

HoodieDataSourceInternalWriter

public HoodieDataSourceInternalWriter(String instantTime,
                                      HoodieWriteConfig writeConfig,
                                      org.apache.spark.sql.types.StructType structType,
                                      org.apache.spark.sql.SparkSession sparkSession,
                                      org.apache.hadoop.conf.Configuration configuration,
                                      org.apache.spark.sql.sources.v2.DataSourceOptions dataSourceOptions,
                                      boolean populateMetaFields,
                                      boolean arePartitionRecordsSorted)

方法详细资料

createWriterFactory

public org.apache.spark.sql.sources.v2.writer.DataWriterFactory<org.apache.spark.sql.catalyst.InternalRow> createWriterFactory()

指定者:

createWriterFactory 在接口中 org.apache.spark.sql.sources.v2.writer.DataSourceWriter

useCommitCoordinator

public boolean useCommitCoordinator()

指定者:

useCommitCoordinator 在接口中 org.apache.spark.sql.sources.v2.writer.DataSourceWriter

onDataWriterCommit

public void onDataWriterCommit(org.apache.spark.sql.sources.v2.writer.WriterCommitMessage message)

指定者:

onDataWriterCommit 在接口中 org.apache.spark.sql.sources.v2.writer.DataSourceWriter

commit

public void commit(org.apache.spark.sql.sources.v2.writer.WriterCommitMessage[] messages)

指定者:

commit 在接口中 org.apache.spark.sql.sources.v2.writer.DataSourceWriter

abort

public void abort(org.apache.spark.sql.sources.v2.writer.WriterCommitMessage[] messages)

指定者:

abort 在接口中 org.apache.spark.sql.sources.v2.writer.DataSourceWriter