org.apache.hudi.hadoop.fs.inline

Class HadoopInLineFSUtils

java.lang.Object

org.apache.hudi.storage.inline.InLineFSUtils

org.apache.hudi.hadoop.fs.inline.HadoopInLineFSUtils

public class HadoopInLineFSUtils
extends InLineFSUtils

Utils to parse InLineFileSystem paths. Inline FS format: "inlinefs:////?start_offset=start_offset>&length=" Eg: "inlinefs:///s3a/?start_offset=20&length=40"

Field Summary

Fields inherited from class org.apache.hudi.storage.inline.InLineFSUtils

EQUALS_STR, LENGTH_STR, LOCAL_FILESYSTEM_SCHEME, SCHEME, SCHEME_SEPARATOR, START_OFFSET_STR

Constructor Summary

Constructors

Constructor and Description
HadoopInLineFSUtils()

Method Summary

Modifier and Type	Method and Description
static StorageConfiguration<org.apache.hadoop.conf.Configuration>	buildInlineConf(StorageConfiguration<org.apache.hadoop.conf.Configuration> storageConf)
static org.apache.hadoop.fs.Path	getOuterFilePathFromInlinePath(org.apache.hadoop.fs.Path inlineFSPath) InlineFS Path format: "inlinefs://path/to/outer/file/outer_file_scheme/?start_offset=start_offset>&length="

Methods inherited from class org.apache.hudi.storage.inline.InLineFSUtils

getInlineFilePath, getOuterFilePathFromInlinePath, length, startOffset

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HadoopInLineFSUtils

public HadoopInLineFSUtils()

Method Detail

buildInlineConf

public static StorageConfiguration<org.apache.hadoop.conf.Configuration> buildInlineConf(StorageConfiguration<org.apache.hadoop.conf.Configuration> storageConf)

getOuterFilePathFromInlinePath

public static org.apache.hadoop.fs.Path getOuterFilePathFromInlinePath(org.apache.hadoop.fs.Path inlineFSPath)

InlineFS Path format: "inlinefs://path/to/outer/file/outer_file_scheme/?start_offset=start_offset>&length="

Outer File Path format: "outer_file_scheme://path/to/outer/file"

Example Input: "inlinefs://file1/s3a/?start_offset=20&length=40". Output: "s3a://file1"

Parameters:

inlineFSPath - InLineFS Path to get the outer file Path

Returns:

Outer file Path from the InLineFS Path