io.delta.kernel.internal

Class TableConfig<T>

Object

io.delta.kernel.internal.TableConfig<T>

public class TableConfig<T>
extends Object

Represents the table properties. Also provides methods to access the property values from the table metadata.

Field Summary

Fields

Modifier and Type	Field and Description
static TableConfig<Integer>	CHECKPOINT_INTERVAL How often to checkpoint the delta log? For every N (this config) commits to the log, we will suggest write out a checkpoint file that can speed up the Delta table state reconstruction.
static TableConfig<Long>	COLUMN_MAPPING_MAX_COLUMN_ID This table property is used to control the maximum column mapping ID.
static TableConfig<ColumnMapping.ColumnMappingMode>	COLUMN_MAPPING_MODE This table property is used to control the column mapping mode.
static TableConfig<java.util.Map<String,String>>	COORDINATED_COMMITS_COORDINATOR_CONF
static TableConfig<java.util.Optional<String>>	COORDINATED_COMMITS_COORDINATOR_NAME
static TableConfig<java.util.Map<String,String>>	COORDINATED_COMMITS_TABLE_CONF
static TableConfig<Boolean>	EXPIRED_LOG_CLEANUP_ENABLED Whether to clean up expired checkpoints and delta logs.
static TableConfig<Boolean>	ICEBERG_COMPAT_V2_ENABLED Table property that enables modifying the table in accordance with the Delta-Iceberg Compatibility V2 protocol.
static TableConfig<java.util.Optional<Long>>	IN_COMMIT_TIMESTAMP_ENABLEMENT_TIMESTAMP This table property is used to track the timestamp at which inCommitTimestamps were enabled.
static TableConfig<java.util.Optional<Long>>	IN_COMMIT_TIMESTAMP_ENABLEMENT_VERSION This table property is used to track the version of the table at which inCommitTimestamps were enabled.
static TableConfig<Boolean>	IN_COMMIT_TIMESTAMPS_ENABLED This table property is used to track the enablement of the inCommitTimestamps.
static TableConfig<Long>	LOG_RETENTION The shortest duration we have to keep delta/checkpoint files around before deleting them.
static TableConfig<Long>	TOMBSTONE_RETENTION The shortest duration we have to keep logically deleted data files around before deleting them physically.

Method Summary

Modifier and Type	Method and Description
T	fromMetadata(java.util.Map<String,String> configuration) Returns the value of the table property from the given configuration.
T	fromMetadata(Metadata metadata) Returns the value of the table property from the given metadata.
String	getKey() Returns the key of the table property.
static java.util.Map<String,String>	validateProperties(java.util.Map<String,String> configurations) Validates that the given properties have the delta prefix in the key name, and they are in the set of valid properties.

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

TOMBSTONE_RETENTION

public static final TableConfig<Long> TOMBSTONE_RETENTION

The shortest duration we have to keep logically deleted data files around before deleting them physically.

Note: this value should be large enough:

• It should be larger than the longest possible duration of a job if you decide to run "VACUUM" when there are concurrent readers or writers accessing the table.
• If you are running a streaming query reading from the table, you should make sure the query doesn't stop longer than this value. Otherwise, the query may not be able to restart as it still needs to read old files.

CHECKPOINT_INTERVAL

public static final TableConfig<Integer> CHECKPOINT_INTERVAL

How often to checkpoint the delta log? For every N (this config) commits to the log, we will suggest write out a checkpoint file that can speed up the Delta table state reconstruction.

LOG_RETENTION

public static final TableConfig<Long> LOG_RETENTION

The shortest duration we have to keep delta/checkpoint files around before deleting them. We can only delete delta files that are before a checkpoint.

EXPIRED_LOG_CLEANUP_ENABLED

public static final TableConfig<Boolean> EXPIRED_LOG_CLEANUP_ENABLED

Whether to clean up expired checkpoints and delta logs.

IN_COMMIT_TIMESTAMPS_ENABLED

public static final TableConfig<Boolean> IN_COMMIT_TIMESTAMPS_ENABLED

This table property is used to track the enablement of the inCommitTimestamps.

When enabled, commit metadata includes a monotonically increasing timestamp that allows for reliable TIMESTAMP AS OF time travel even if filesystem operations change a commit file's modification timestamp.

IN_COMMIT_TIMESTAMP_ENABLEMENT_VERSION

public static final TableConfig<java.util.Optional<Long>> IN_COMMIT_TIMESTAMP_ENABLEMENT_VERSION

This table property is used to track the version of the table at which inCommitTimestamps were enabled.

IN_COMMIT_TIMESTAMP_ENABLEMENT_TIMESTAMP

public static final TableConfig<java.util.Optional<Long>> IN_COMMIT_TIMESTAMP_ENABLEMENT_TIMESTAMP

This table property is used to track the timestamp at which inCommitTimestamps were enabled. More specifically, it is the inCommitTimestamps of the commit with the version specified in IN_COMMIT_TIMESTAMP_ENABLEMENT_VERSION.

COORDINATED_COMMITS_COORDINATOR_NAME

public static final TableConfig<java.util.Optional<String>> COORDINATED_COMMITS_COORDINATOR_NAME

COORDINATED_COMMITS_COORDINATOR_CONF

public static final TableConfig<java.util.Map<String,String>> COORDINATED_COMMITS_COORDINATOR_CONF

COORDINATED_COMMITS_TABLE_CONF

public static final TableConfig<java.util.Map<String,String>> COORDINATED_COMMITS_TABLE_CONF

COLUMN_MAPPING_MODE

public static final TableConfig<ColumnMapping.ColumnMappingMode> COLUMN_MAPPING_MODE

This table property is used to control the column mapping mode.

COLUMN_MAPPING_MAX_COLUMN_ID

public static final TableConfig<Long> COLUMN_MAPPING_MAX_COLUMN_ID

This table property is used to control the maximum column mapping ID.

ICEBERG_COMPAT_V2_ENABLED

public static final TableConfig<Boolean> ICEBERG_COMPAT_V2_ENABLED

Table property that enables modifying the table in accordance with the Delta-Iceberg Compatibility V2 protocol.

See Also:

Delta-Iceberg Compatibility V2 Protocol

Method Detail

validateProperties

public static java.util.Map<String,String> validateProperties(java.util.Map<String,String> configurations)

Validates that the given properties have the delta prefix in the key name, and they are in the set of valid properties. The caller should get the validated configurations and store the case of the property name defined in TableConfig.

Parameters:

configurations - the properties to validate

Throws:

InvalidConfigurationValueException - if any of the properties are invalid

UnknownConfigurationException - if any of the properties are unknown

fromMetadata

public T fromMetadata(Metadata metadata)

Returns the value of the table property from the given metadata.

Parameters:

metadata - the table metadata

Returns:

the value of the table property

fromMetadata

public T fromMetadata(java.util.Map<String,String> configuration)

Returns the value of the table property from the given configuration.

Parameters:

configuration - the table configuration

Returns:

the value of the table property

getKey

public String getKey()

Returns the key of the table property.

Returns:

the key of the table property