org.apache.hudi.common.util

Class FileIOUtils

java.lang.Object

org.apache.hudi.common.util.FileIOUtils

public class FileIOUtils
extends Object

Bunch of utility methods for working with files and byte streams.

Field Summary

Fields

Modifier and Type	Field and Description
static long	KB
static org.apache.log4j.Logger	LOG

Constructor Summary

Constructors

Constructor and Description
FileIOUtils()

Method Summary

Modifier and Type	Method and Description
static void	closeQuietly(Closeable closeable) Closes Closeable quietly.
static void	copy(org.apache.hadoop.fs.FileSystem fileSystem, org.apache.hadoop.fs.Path sourceFilePath, org.apache.hadoop.fs.Path destFilePath) Copies the file content from source path to destination path.
static void	copy(InputStream inputStream, OutputStream outputStream)
static void	createFileInPath(org.apache.hadoop.fs.FileSystem fileSystem, org.apache.hadoop.fs.Path fullPath, Option<byte[]> content)
static void	createFileInPath(org.apache.hadoop.fs.FileSystem fileSystem, org.apache.hadoop.fs.Path fullPath, Option<byte[]> content, boolean ignoreIOE)
static void	deleteDirectory(File directory)
static String[]	getConfiguredLocalDirs() Return the configured local directories where hudi can write files.
static void	mkdir(File directory)
static byte[]	readAsByteArray(InputStream input)
static byte[]	readAsByteArray(InputStream input, int outputSize)
static String	readAsUTFString(InputStream input)
static String	readAsUTFString(InputStream input, int length)
static List<String>	readAsUTFStringLines(InputStream input) Reads the input stream into String lines.
static Option<byte[]>	readDataFromPath(org.apache.hadoop.fs.FileSystem fileSystem, org.apache.hadoop.fs.Path detailPath)
static Option<byte[]>	readDataFromPath(org.apache.hadoop.fs.FileSystem fileSystem, org.apache.hadoop.fs.Path detailPath, boolean ignoreIOE)
static void	writeStringToFile(String str, String filePath)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

public static final org.apache.log4j.Logger LOG

KB

public static final long KB

See Also:

Constant Field Values

Constructor Detail

FileIOUtils

public FileIOUtils()

Method Detail

deleteDirectory

public static void deleteDirectory(File directory)
                            throws IOException

Throws:

IOException

mkdir

public static void mkdir(File directory)
                  throws IOException

Throws:

IOException

readAsUTFString

public static String readAsUTFString(InputStream input)
                              throws IOException

Throws:

IOException

readAsUTFString

public static String readAsUTFString(InputStream input,
                                     int length)
                              throws IOException

Throws:

IOException

readAsUTFStringLines

public static List<String> readAsUTFStringLines(InputStream input)

Reads the input stream into String lines.

Parameters:

input - InputStream instance.

Returns:

String lines in a list.

copy

public static void copy(InputStream inputStream,
                        OutputStream outputStream)
                 throws IOException

Throws:

IOException

copy

public static void copy(org.apache.hadoop.fs.FileSystem fileSystem,
                        org.apache.hadoop.fs.Path sourceFilePath,
                        org.apache.hadoop.fs.Path destFilePath)

Copies the file content from source path to destination path.

Parameters:

fileSystem - FileSystem instance.

sourceFilePath - Source file path.

destFilePath - Destination file path.

readAsByteArray

public static byte[] readAsByteArray(InputStream input)
                              throws IOException

Throws:

IOException

readAsByteArray

public static byte[] readAsByteArray(InputStream input,
                                     int outputSize)
                              throws IOException

Throws:

IOException

writeStringToFile

public static void writeStringToFile(String str,
                                     String filePath)
                              throws IOException

Throws:

IOException

closeQuietly

public static void closeQuietly(Closeable closeable)

Closes Closeable quietly.

Parameters:

closeable - Closeable to close

createFileInPath

public static void createFileInPath(org.apache.hadoop.fs.FileSystem fileSystem,
                                    org.apache.hadoop.fs.Path fullPath,
                                    Option<byte[]> content,
                                    boolean ignoreIOE)

createFileInPath

public static void createFileInPath(org.apache.hadoop.fs.FileSystem fileSystem,
                                    org.apache.hadoop.fs.Path fullPath,
                                    Option<byte[]> content)

readDataFromPath

public static Option<byte[]> readDataFromPath(org.apache.hadoop.fs.FileSystem fileSystem,
                                              org.apache.hadoop.fs.Path detailPath,
                                              boolean ignoreIOE)

readDataFromPath

public static Option<byte[]> readDataFromPath(org.apache.hadoop.fs.FileSystem fileSystem,
                                              org.apache.hadoop.fs.Path detailPath)

getConfiguredLocalDirs

public static String[] getConfiguredLocalDirs()

Return the configured local directories where hudi can write files. This method does not create any directories on its own, it only encapsulates the logic of locating the local directories according to deployment mode.