io.delta.kernel.internal.util

Class SchemaUtils

Object

io.delta.kernel.internal.util.SchemaUtils

public class SchemaUtils
extends Object

Utility methods for schema related operations such as validating the schema has no duplicate columns and the names contain only valid characters.

Method Summary

Modifier and Type	Method and Description
static java.util.Map<String,Literal>	casePreservingPartitionColNames(java.util.List<String> partitionColNames, java.util.Map<String,Literal> partitionValues) Convert the partition column names in partitionValues map into the same case as the column in the table metadata.
static java.util.List<String>	casePreservingPartitionColNames(StructType tableSchema, java.util.List<String> partitionColumns) Delta expects partition column names to be same case preserving as the name in the schema.
static int	findColIndex(StructType schema, String colName) Search (case-insensitive) for the given colName in the schema and return its position in the schema.
static void	validatePartitionColumns(StructType schema, java.util.List<String> partitionCols) Verify the partition columns exists in the table schema and a supported data type for a partition column.
static void	validateSchema(StructType schema, boolean isColumnMappingEnabled) Validate the schema.

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

validateSchema

public static void validateSchema(StructType schema,
                                  boolean isColumnMappingEnabled)

Validate the schema. This method checks if the schema has no duplicate columns, the names contain only valid characters and the data types are supported.

Parameters:

schema - the schema to validate

isColumnMappingEnabled - whether column mapping is enabled. When column mapping is enabled, the column names in the schema can contain special characters that are allowed as column names in the Parquet file

Throws:

IllegalArgumentException - if the schema is invalid

validatePartitionColumns

public static void validatePartitionColumns(StructType schema,
                                            java.util.List<String> partitionCols)

Verify the partition columns exists in the table schema and a supported data type for a partition column.

Parameters:

schema -

partitionCols -

casePreservingPartitionColNames

public static java.util.List<String> casePreservingPartitionColNames(StructType tableSchema,
                                                                     java.util.List<String> partitionColumns)

Delta expects partition column names to be same case preserving as the name in the schema. E.g: Schema: (a INT, B STRING) and partition columns: (b). In this case we store the schema as (a INT, B STRING) and partition columns as (B).

This method expects the inputs are already validated (i.e. schema contains all the partition columns).

casePreservingPartitionColNames

public static java.util.Map<String,Literal> casePreservingPartitionColNames(java.util.List<String> partitionColNames,
                                                                            java.util.Map<String,Literal> partitionValues)

Convert the partition column names in partitionValues map into the same case as the column in the table metadata. Delta expects the partition column names to preserve the case same as the table schema.

Parameters:

partitionColNames - List of partition columns in the table metadata. The names preserve the case as given by the connector when the table is created.

partitionValues - Map of partition column name to partition value. Convert the partition column name to be same case preserving name as its equivalent column in the partitionColName. Column name comparison is case-insensitive.

Returns:

Rewritten partitionValues map with names case preserved.

findColIndex

public static int findColIndex(StructType schema,
                               String colName)

Search (case-insensitive) for the given colName in the schema and return its position in the schema.

Parameters:

schema - StructType

colName - Name of the column whose index is needed.

Returns:

Valid index or -1 if not found.