org.apache.hudi.common.util

Class InternalSchemaCache

java.lang.Object

org.apache.hudi.common.util.InternalSchemaCache

public class InternalSchemaCache
extends Object

Constructor Summary

Constructors

Constructor and Description
InternalSchemaCache()

Method Summary

Modifier and Type	Method and Description
static Pair<Option<String>,Option<String>>	getInternalSchemaAndAvroSchemaForClusteringAndCompaction(HoodieTableMetaClient metaClient, String compactionAndClusteringInstant) Get internalSchema and avroSchema for compaction/cluster operation.
static InternalSchema	getInternalSchemaByVersionId(long versionId, String tablePath, org.apache.hadoop.conf.Configuration hadoopConf, String validCommits) Give a schema versionId return its internalSchema.
static InternalSchema	searchSchemaAndCache(long versionID, HoodieTableMetaClient metaClient, boolean cacheEnable) Search internalSchema based on versionID.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

InternalSchemaCache

public InternalSchemaCache()

Method Detail

searchSchemaAndCache

public static InternalSchema searchSchemaAndCache(long versionID,
                                                  HoodieTableMetaClient metaClient,
                                                  boolean cacheEnable)

Search internalSchema based on versionID. first step: try to get internalSchema from hoodie commit files, we no need to add lock. if we cannot get internalSchema by first step, then we try to get internalSchema from cache.

Parameters:

versionID - schema version_id need to search

metaClient - current hoodie metaClient

Returns:

internalSchema

getInternalSchemaAndAvroSchemaForClusteringAndCompaction

public static Pair<Option<String>,Option<String>> getInternalSchemaAndAvroSchemaForClusteringAndCompaction(HoodieTableMetaClient metaClient,
                                                                                                           String compactionAndClusteringInstant)

Get internalSchema and avroSchema for compaction/cluster operation.

Parameters:

metaClient - current hoodie metaClient

compactionAndClusteringInstant - first instant before current compaction/cluster instant

Returns:

(internalSchemaStrOpt, avroSchemaStrOpt) a pair of InternalSchema/avroSchema

getInternalSchemaByVersionId

public static InternalSchema getInternalSchemaByVersionId(long versionId,
                                                          String tablePath,
                                                          org.apache.hadoop.conf.Configuration hadoopConf,
                                                          String validCommits)

Give a schema versionId return its internalSchema. This method will be called by spark tasks, we should minimize time cost. We try our best to not use metaClient， since the initialization of metaClient is time cost step1： try to parser internalSchema from HoodieInstant directly step2： if we cannot parser internalSchema in step1， try to find internalSchema in historySchema.

Parameters:

versionId - the internalSchema version to be search.

tablePath - table path

hadoopConf - conf

validCommits - current validate commits, use to make up the commit file path/verify the validity of the history schema files

Returns:

a internalSchema.