org.apache.iceberg.spark

Class FileRewriteCoordinator

java.lang.Object

org.apache.iceberg.spark.FileRewriteCoordinator

public class FileRewriteCoordinator
extends java.lang.Object

Method Summary

Modifier and Type	Method and Description
void	clearRewrite(org.apache.iceberg.Table table, java.lang.String fileSetId)
java.util.Set<org.apache.iceberg.DataFile>	fetchNewDataFiles(org.apache.iceberg.Table table, java.lang.String fileSetId) Deprecated. will be removed in 1.4.0; use fetchNewFiles(Table, String) instead.
java.util.Set<F>	fetchNewFiles(org.apache.iceberg.Table table, java.lang.String fileSetId)
java.util.Set<java.lang.String>	fetchSetIds(org.apache.iceberg.Table table)
java.util.Set<java.lang.String>	fetchSetIDs(org.apache.iceberg.Table table) Deprecated. will be removed in 1.4.0; use fetchSetIds(Table) instead
static FileRewriteCoordinator	get()
void	stageRewrite(org.apache.iceberg.Table table, java.lang.String fileSetId, java.util.Set<F> newFiles) Called to persist the output of a rewrite action for a specific group.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

get

public static FileRewriteCoordinator get()

fetchNewDataFiles

@Deprecated
public java.util.Set<org.apache.iceberg.DataFile> fetchNewDataFiles(org.apache.iceberg.Table table,
                                                                                java.lang.String fileSetId)

Deprecated. will be removed in 1.4.0; use fetchNewFiles(Table, String) instead.

fetchSetIDs

@Deprecated
public java.util.Set<java.lang.String> fetchSetIDs(org.apache.iceberg.Table table)

Deprecated. will be removed in 1.4.0; use fetchSetIds(Table) instead

stageRewrite

public void stageRewrite(org.apache.iceberg.Table table,
                         java.lang.String fileSetId,
                         java.util.Set<F> newFiles)

Called to persist the output of a rewrite action for a specific group. Since the write is done via a Spark Datasource, we have to propagate the result through this side-effect call.

Parameters:

table - table where the rewrite is occurring

fileSetId - the id used to identify the source set of files being rewritten

newFiles - the new files which have been written

fetchNewFiles

public java.util.Set<F> fetchNewFiles(org.apache.iceberg.Table table,
                                      java.lang.String fileSetId)

clearRewrite

public void clearRewrite(org.apache.iceberg.Table table,
                         java.lang.String fileSetId)

fetchSetIds

public java.util.Set<java.lang.String> fetchSetIds(org.apache.iceberg.Table table)