Package org.apache.pinot.segment.local.segment.index.loader

Class LoaderUtils

java.lang.Object

org.apache.pinot.segment.local.segment.index.loader.LoaderUtils

public class LoaderUtils
extends Object

Method Summary

Modifier and Type	Method	Description
static List<String>	getStringListFromSegmentProperties(String key, org.apache.commons.configuration.PropertiesConfiguration segmentProperties)	Get string list from segment properties.
static void	reloadFailureRecovery(File indexDir)	Try to recover a segment from reload failures (reloadSegment() method in HelixInstanceDataManager).
static void	writeIndexToV3Format(SegmentDirectory.Writer segmentWriter, String column, File indexFile, IndexType<?,?,?> indexType)	Write an index file to v3 format single index file and remove the old one.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

writeIndexToV3Format

public static void writeIndexToV3Format(SegmentDirectory.Writer segmentWriter,
                                        String column,
                                        File indexFile,
                                        IndexType<?,?,?> indexType)
                                 throws IOException

Write an index file to v3 format single index file and remove the old one.

Parameters:

segmentWriter - v3 format segment writer.

column - column name.

indexFile - index file to write from.

indexType - index type.

Throws:

IOException

getStringListFromSegmentProperties

public static List<String> getStringListFromSegmentProperties(String key,
                                                              org.apache.commons.configuration.PropertiesConfiguration segmentProperties)

Get string list from segment properties.

NOTE: When the property associated with the key is empty, AbstractConfiguration.getList(String) will return an empty string singleton list. Using this method will return an empty list instead.

Parameters:

key - property key.

Returns:

string list value for the property.

reloadFailureRecovery

public static void reloadFailureRecovery(File indexDir)
                                  throws IOException

Try to recover a segment from reload failures (reloadSegment() method in HelixInstanceDataManager). This has no effect for normal segments.

Reload failures include normal failures like Java exceptions (called in reloadSegment() finally block) and hard failures such as server restart during reload and JVM crash (called before trying to load segment from the index directory).

The following failure scenarios could happen (use atomic renaming operation to classify scenarios):

• Failure happens before renaming index directory to segment backup directory:

  Only index directory exists. No need to recover because we have not loaded segment so index directory is left unchanged.

• Failure happens before renaming segment backup directory to segment temporary directory:

  Segment backup directory exists, and index directory might exist. Index directory could be left in corrupted state because we tried to load segment from it and potentially added indexes. Need to recover index directory from segment backup directory.

• Failure happens after renaming segment backup directory to segment temporary directory (during deleting segment temporary directory):

  Index directory and segment temporary directory exist. Segment has been successfully loaded, so index segment is in good state. Delete segment temporary directory.

Should be called before trying to load the segment or metadata from index directory.

Throws:

IOException