tech.mlsql.common.utils.hdfs

Class HdfsClassLoader

java.lang.Object

java.lang.ClassLoader

tech.mlsql.common.utils.hdfs.HdfsClassLoader

public class HdfsClassLoader
extends ClassLoader

A ClassLoader that reads its class data from a jar file stored in HDFS.

Author:

jeff@opower.com

Field Summary

Fields

Modifier and Type	Field and Description
static String	ATTEMPT_LOCAL_LOAD_FIRST A configuration option to indicate whether or not to skip the standard ClassLoader hierarchical search and look at the configured jar first.

Constructor Summary

Constructors

Constructor and Description
HdfsClassLoader(org.apache.hadoop.conf.Configuration configuration, org.apache.hadoop.fs.Path jar)

Method Summary

Modifier and Type	Method and Description
Class	findClass(String className) Search for the class in the configured jar file stored in HDFS.
Class	loadClass(String className) Override to allow for checking the local jar first instead of the standard search which would check the parent class loader first.

Methods inherited from class java.lang.ClassLoader

clearAssertionStatus, defineClass, defineClass, defineClass, defineClass, definePackage, findLibrary, findLoadedClass, findResource, findResources, findSystemClass, getClassLoadingLock, getPackage, getPackages, getParent, getResource, getResourceAsStream, getResources, getSystemClassLoader, getSystemResource, getSystemResourceAsStream, getSystemResources, loadClass, registerAsParallelCapable, resolveClass, setClassAssertionStatus, setDefaultAssertionStatus, setPackageAssertionStatus, setSigners

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

ATTEMPT_LOCAL_LOAD_FIRST

public static final String ATTEMPT_LOCAL_LOAD_FIRST

A configuration option to indicate whether or not to skip the standard ClassLoader hierarchical search and look at the configured jar first. Defaults to false so that the standard search is performed, but it may be helpful to turn to true for debugging.

See Also:

Constant Field Values

Constructor Detail

HdfsClassLoader

public HdfsClassLoader(org.apache.hadoop.conf.Configuration configuration,
                       org.apache.hadoop.fs.Path jar)

Parameters:

configuration - The Hadoop configuration to use to read from HDFS

jar - A path to a jar file containing classes to load

Method Detail

loadClass

public Class loadClass(String className)
                throws ClassNotFoundException

Override to allow for checking the local jar first instead of the standard search which would check the parent class loader first.

Overrides:

loadClass in class ClassLoader

Throws:

ClassNotFoundException

findClass

public Class findClass(String className)
                throws ClassNotFoundException

Search for the class in the configured jar file stored in HDFS.

Overrides:

findClass in class ClassLoader

Throws:

ClassNotFoundException