org.apache.iceberg.gcp.gcs

Class GCSFileIO

java.lang.Object

org.apache.iceberg.gcp.gcs.GCSFileIO

All Implemented Interfaces:

java.io.Closeable, java.io.Serializable, java.lang.AutoCloseable, org.apache.iceberg.io.DelegateFileIO, org.apache.iceberg.io.FileIO, org.apache.iceberg.io.SupportsBulkOperations, org.apache.iceberg.io.SupportsPrefixOperations

public class GCSFileIO
extends java.lang.Object
implements org.apache.iceberg.io.DelegateFileIO

FileIO Implementation backed by Google Cloud Storage (GCS)

Locations follow the conventions used by BlobId.fromGsUtilUri that follow the convention

 gs://<bucket>/<blob_path>

See Cloud Storage Overview

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
GCSFileIO() No-arg constructor to load the FileIO dynamically.
GCSFileIO(org.apache.iceberg.util.SerializableSupplier<com.google.cloud.storage.Storage> storageSupplier, GCPProperties gcpProperties) Constructor with custom storage supplier and GCP properties.

Method Summary

Modifier and Type	Method and Description
com.google.cloud.storage.Storage	client()
void	close()
void	deleteFile(java.lang.String path)
void	deleteFiles(java.lang.Iterable<java.lang.String> pathsToDelete)
void	deletePrefix(java.lang.String prefix)
void	initialize(java.util.Map<java.lang.String,java.lang.String> props)
java.lang.Iterable<org.apache.iceberg.io.FileInfo>	listPrefix(java.lang.String prefix)
org.apache.iceberg.io.InputFile	newInputFile(java.lang.String path)
org.apache.iceberg.io.InputFile	newInputFile(java.lang.String path, long length)
org.apache.iceberg.io.OutputFile	newOutputFile(java.lang.String path)
java.util.Map<java.lang.String,java.lang.String>	properties()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.iceberg.io.FileIO

deleteFile, deleteFile

Constructor Detail

GCSFileIO

public GCSFileIO()

No-arg constructor to load the FileIO dynamically.

All fields are initialized by calling initialize(Map) later.

GCSFileIO

public GCSFileIO(org.apache.iceberg.util.SerializableSupplier<com.google.cloud.storage.Storage> storageSupplier,
                 GCPProperties gcpProperties)

Constructor with custom storage supplier and GCP properties.

Calling initialize(Map) will overwrite information set in this constructor.

Parameters:

storageSupplier - storage supplier

gcpProperties - gcp properties

Method Detail

newInputFile

public org.apache.iceberg.io.InputFile newInputFile(java.lang.String path)

Specified by:

newInputFile in interface org.apache.iceberg.io.FileIO

newInputFile

public org.apache.iceberg.io.InputFile newInputFile(java.lang.String path,
                                                    long length)

Specified by:

newInputFile in interface org.apache.iceberg.io.FileIO

newOutputFile

public org.apache.iceberg.io.OutputFile newOutputFile(java.lang.String path)

Specified by:

newOutputFile in interface org.apache.iceberg.io.FileIO

deleteFile

public void deleteFile(java.lang.String path)

Specified by:

deleteFile in interface org.apache.iceberg.io.FileIO

properties

public java.util.Map<java.lang.String,java.lang.String> properties()

Specified by:

properties in interface org.apache.iceberg.io.FileIO

client

public com.google.cloud.storage.Storage client()

initialize

public void initialize(java.util.Map<java.lang.String,java.lang.String> props)

Specified by:

initialize in interface org.apache.iceberg.io.FileIO

close

public void close()

Specified by:

close in interface java.io.Closeable

Specified by:

close in interface java.lang.AutoCloseable

Specified by:

close in interface org.apache.iceberg.io.FileIO

listPrefix

public java.lang.Iterable<org.apache.iceberg.io.FileInfo> listPrefix(java.lang.String prefix)

Specified by:

listPrefix in interface org.apache.iceberg.io.SupportsPrefixOperations

deletePrefix

public void deletePrefix(java.lang.String prefix)

Specified by:

deletePrefix in interface org.apache.iceberg.io.SupportsPrefixOperations

deleteFiles

public void deleteFiles(java.lang.Iterable<java.lang.String> pathsToDelete)
                 throws org.apache.iceberg.io.BulkDeletionFailureException

Specified by:

deleteFiles in interface org.apache.iceberg.io.SupportsBulkOperations

Throws:

org.apache.iceberg.io.BulkDeletionFailureException