com.aliyun.odps.io

Class IoUtils

java.lang.Object

com.aliyun.odps.io.IoUtils

public class IoUtils
extends Object

Constructor Summary

Constructors

Constructor and Description
IoUtils()

Method Summary

Modifier and Type	Method and Description
static String	getMetaFileLocation(String externalTableLocation) If data has been output by ODPS through external table, this returns the expected location of the meta file generated by ODPS, see parseOutputSubDirectoriesFromMeta(String) for getting the list of valid subdirectories output from meta file.
static String	getSystemOutputDirectory(String externalTableLocation) Get the system directory ODPS used when outputting unstructured data
static String[]	parseOutputSubDirectoriesFromMeta(String metaContent) Get valid directories for unstructured data output by ODPS.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

IoUtils

public IoUtils()

Method Detail

getMetaFileLocation

public static String getMetaFileLocation(String externalTableLocation)

If data has been output by ODPS through external table, this returns the expected location of the meta file generated by ODPS, see parseOutputSubDirectoriesFromMeta(String) for getting the list of valid subdirectories output from meta file.

Parameters:

externalTableLocation - the location specified when creating external table

Returns:

the expected meta file location

getSystemOutputDirectory

public static String getSystemOutputDirectory(String externalTableLocation)

Get the system directory ODPS used when outputting unstructured data

Parameters:

externalTableLocation - the location specified when creating external table

Returns:

the directory in location path used for system output

parseOutputSubDirectoriesFromMeta

public static String[] parseOutputSubDirectoriesFromMeta(String metaContent)

Get valid directories for unstructured data output by ODPS. Note: most of time system output directory will only include valid data (i.e., valid subdirectories) however, when an output job was interrupted (failed or killed), it is possible some invalid data will be left-over. In those cases, meta content provides the ground-truth as to which directories are complete and valid.

Parameters:

metaContent - file content read from ODPS meta file

Returns:

array of directories (inside the ODPS system directory)