org.apache.hudi.common.fs.inline

Class InLineFSUtils

java.lang.Object

org.apache.hudi.common.fs.inline.InLineFSUtils

public class InLineFSUtils
extends Object

Utils to parse InLineFileSystem paths. Inline FS format: "inlinefs:////?start_offset=start_offset>&length=" Eg: "inlinefs:///s3a/?start_offset=20&length=40"

Constructor Summary

Constructors

Constructor and Description
InLineFSUtils()

Method Summary

Modifier and Type	Method and Description
static org.apache.hadoop.fs.Path	getInlineFilePath(org.apache.hadoop.fs.Path outerPath, String origScheme, long inLineStartOffset, long inLineLength) Get the InlineFS Path for a given schema and its Path.
static org.apache.hadoop.fs.Path	getOuterFilePathFromInlinePath(org.apache.hadoop.fs.Path inlineFSPath) InlineFS Path format: "inlinefs://path/to/outer/file/outer_file_scheme/?start_offset=start_offset>&length="
static long	length(org.apache.hadoop.fs.Path inlinePath) Returns length of the block (embedded w/in the base file) identified by the given InlineFS path input: "inlinefs:/file1/s3a/?start_offset=20&length=40".
static long	startOffset(org.apache.hadoop.fs.Path inlineFSPath) Returns start offset w/in the base for the block identified by the given InlineFS path input: "inlinefs://file1/s3a/?start_offset=20&length=40".

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

InLineFSUtils

public InLineFSUtils()

Method Detail

getInlineFilePath

public static org.apache.hadoop.fs.Path getInlineFilePath(org.apache.hadoop.fs.Path outerPath,
                                                          String origScheme,
                                                          long inLineStartOffset,
                                                          long inLineLength)

Get the InlineFS Path for a given schema and its Path.

Examples: Input Path: s3a://file1, origScheme: file, startOffset = 20, length = 40 Output: "inlinefs://file1/s3a/?start_offset=20&length=40"

Parameters:

outerPath - The outer file Path

origScheme - The file schema

inLineStartOffset - Start offset for the inline file

inLineLength - Length for the inline file

Returns:

InlineFS Path for the requested outer path and schema

getOuterFilePathFromInlinePath

public static org.apache.hadoop.fs.Path getOuterFilePathFromInlinePath(org.apache.hadoop.fs.Path inlineFSPath)

InlineFS Path format: "inlinefs://path/to/outer/file/outer_file_scheme/?start_offset=start_offset>&length="

Outer File Path format: "outer_file_scheme://path/to/outer/file"

Example Input: "inlinefs://file1/s3a/?start_offset=20&length=40". Output: "s3a://file1"

Parameters:

inlineFSPath - InLineFS Path to get the outer file Path

Returns:

Outer file Path from the InLineFS Path

startOffset

public static long startOffset(org.apache.hadoop.fs.Path inlineFSPath)

Returns start offset w/in the base for the block identified by the given InlineFS path input: "inlinefs://file1/s3a/?start_offset=20&length=40". output: 20

length

public static long length(org.apache.hadoop.fs.Path inlinePath)

Returns length of the block (embedded w/in the base file) identified by the given InlineFS path input: "inlinefs:/file1/s3a/?start_offset=20&length=40". output: 40