org.apache.iceberg.spark.source

Class SparkPartitionedFanoutWriter

java.lang.Object

org.apache.iceberg.io.BaseTaskWriter<T>

org.apache.iceberg.io.PartitionedFanoutWriter<org.apache.spark.sql.catalyst.InternalRow>

org.apache.iceberg.spark.source.SparkPartitionedFanoutWriter

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable, org.apache.iceberg.io.TaskWriter<org.apache.spark.sql.catalyst.InternalRow>

public class SparkPartitionedFanoutWriter
extends org.apache.iceberg.io.PartitionedFanoutWriter<org.apache.spark.sql.catalyst.InternalRow>

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.iceberg.io.BaseTaskWriter

org.apache.iceberg.io.BaseTaskWriter.BaseEqualityDeltaWriter, org.apache.iceberg.io.BaseTaskWriter.RollingEqDeleteWriter, org.apache.iceberg.io.BaseTaskWriter.RollingFileWriter

Constructor Summary

Constructors

Constructor and Description
SparkPartitionedFanoutWriter(org.apache.iceberg.PartitionSpec spec, org.apache.iceberg.FileFormat format, org.apache.iceberg.io.FileAppenderFactory<org.apache.spark.sql.catalyst.InternalRow> appenderFactory, org.apache.iceberg.io.OutputFileFactory fileFactory, org.apache.iceberg.io.FileIO io, long targetFileSize, org.apache.iceberg.Schema schema, org.apache.spark.sql.types.StructType sparkSchema)

Method Summary

Modifier and Type	Method and Description
protected org.apache.iceberg.PartitionKey	partition(org.apache.spark.sql.catalyst.InternalRow row)

Methods inherited from class org.apache.iceberg.io.PartitionedFanoutWriter

close, write

Methods inherited from class org.apache.iceberg.io.BaseTaskWriter

abort, complete, setFailure, spec

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.iceberg.io.TaskWriter

dataFiles

Constructor Detail

SparkPartitionedFanoutWriter

public SparkPartitionedFanoutWriter(org.apache.iceberg.PartitionSpec spec,
                                    org.apache.iceberg.FileFormat format,
                                    org.apache.iceberg.io.FileAppenderFactory<org.apache.spark.sql.catalyst.InternalRow> appenderFactory,
                                    org.apache.iceberg.io.OutputFileFactory fileFactory,
                                    org.apache.iceberg.io.FileIO io,
                                    long targetFileSize,
                                    org.apache.iceberg.Schema schema,
                                    org.apache.spark.sql.types.StructType sparkSchema)

Method Detail

partition

protected org.apache.iceberg.PartitionKey partition(org.apache.spark.sql.catalyst.InternalRow row)

Specified by:

partition in class org.apache.iceberg.io.PartitionedFanoutWriter<org.apache.spark.sql.catalyst.InternalRow>