A B C D E F G H I J L M N O P R S T V X
All Classes All Packages

A

AddPartition - Class in net.snowflake.hivemetastoreconnector.commands

A class for the AddPartition command

AddPartition(Table, Iterator<Partition>, SnowflakeConf, Configuration, boolean) - Constructor for class net.snowflake.hivemetastoreconnector.commands.AddPartition

Creates an AddPartition command without an event

AddPartition(AddPartitionEvent, SnowflakeConf) - Constructor for class net.snowflake.hivemetastoreconnector.commands.AddPartition

Creates an AddPartition command

AddPartition(AlterPartitionEvent, SnowflakeConf) - Constructor for class net.snowflake.hivemetastoreconnector.commands.AddPartition

Creates an AddPartition command

AlterExternalTable - Class in net.snowflake.hivemetastoreconnector.commands

A class for the AlterExternalTable command TODO: Split into TouchExternalTable and AlterColumnsExternalTable

AlterExternalTable(AlterTableEvent, SnowflakeConf) - Constructor for class net.snowflake.hivemetastoreconnector.commands.AlterExternalTable

Creates a AlterTable command

AVRO - net.snowflake.hivemetastoreconnector.util.HiveToSnowflakeType.SnowflakeFileFormatType

AWS - net.snowflake.hivemetastoreconnector.util.StageCredentialUtil.StageCredentialType

B

BY_VARNAME - Static variable in enum net.snowflake.hivemetastoreconnector.SnowflakeConf.ConfVars

C

Command - Class in net.snowflake.hivemetastoreconnector.commands

A class that represents a command to generate executable Snowflake statements from

Command(String, String) - Constructor for class net.snowflake.hivemetastoreconnector.commands.Command

Command(Table) - Constructor for class net.snowflake.hivemetastoreconnector.commands.Command

CommandGenerator - Class in net.snowflake.hivemetastoreconnector.core

A class that generates the commands to be executed

CommandGenerator() - Constructor for class net.snowflake.hivemetastoreconnector.core.CommandGenerator

compact(List<AddPartition>) - Static method in class net.snowflake.hivemetastoreconnector.commands.AddPartition

Combines N add partition commands with M total partitions into ceil(M / MAX_ADDED_PARTITIONS) new add partition commands.

createAndExecuteCommandForSnowflake(ListenerEvent, SnowflakeConf) - Static method in class net.snowflake.hivemetastoreconnector.core.SnowflakeClient

Creates and executes an event for snowflake.

CreateExternalTable - Class in net.snowflake.hivemetastoreconnector.commands

A class for the CreateExternalTable command

CreateExternalTable(Table, SnowflakeConf, Configuration, boolean) - Constructor for class net.snowflake.hivemetastoreconnector.commands.CreateExternalTable

Creates a CreateExternalTable command, without an event

CreateExternalTable(CreateTableEvent, SnowflakeConf) - Constructor for class net.snowflake.hivemetastoreconnector.commands.CreateExternalTable

Creates a CreateExternalTable command

CSV - net.snowflake.hivemetastoreconnector.util.HiveToSnowflakeType.SnowflakeFileFormatType

D

DropExternalTable - Class in net.snowflake.hivemetastoreconnector.commands

A class for the DropExternalTable command

DropExternalTable(DropTableEvent, SnowflakeConf) - Constructor for class net.snowflake.hivemetastoreconnector.commands.DropExternalTable

Creates a DropExternalTable command

DropPartition - Class in net.snowflake.hivemetastoreconnector.commands

A class for the DropPartition command

DropPartition(Table, Iterator<String>) - Constructor for class net.snowflake.hivemetastoreconnector.commands.DropPartition

Creates a DropPartition command

DropPartition(DropPartitionEvent) - Constructor for class net.snowflake.hivemetastoreconnector.commands.DropPartition

Creates a DropPartition command

E

enqueueMessage(Command) - Method in class net.snowflake.hivemetastoreconnector.core.Scheduler

Enqueues a message to be collected and batched

escapeSqlComment(String) - Static method in class net.snowflake.hivemetastoreconnector.util.StringUtil

Helper method to escape text in a SQL comment

escapeSqlDataTypeSpec(String) - Static method in class net.snowflake.hivemetastoreconnector.util.StringUtil

Helper method to escape text that's in a data type spec

escapeSqlIdentifier(String) - Static method in class net.snowflake.hivemetastoreconnector.util.StringUtil

Helper method to escape SQL text that's an identifier

escapeSqlText(String) - Static method in class net.snowflake.hivemetastoreconnector.util.StringUtil

Helper method to escape SQL text

executeStatement(Connection, String, SnowflakeConf) - Static method in class net.snowflake.hivemetastoreconnector.core.SnowflakeClient

(Deprecated) Utility method to connect to Snowflake and execute a query.

executeStatements(List<String>, SnowflakeConf, String) - Static method in class net.snowflake.hivemetastoreconnector.core.SnowflakeClient

Helper method to connect to Snowflake and execute a list of queries

F

findByName(String) - Static method in enum net.snowflake.hivemetastoreconnector.SnowflakeConf.ConfVars

G

generateAndExecuteSnowflakeStatements(Command, SnowflakeConf) - Static method in class net.snowflake.hivemetastoreconnector.core.SnowflakeClient

Helper method.

generateColumnStr(FieldSchema, int, HiveToSnowflakeType.SnowflakeFileFormatType, SnowflakeConf) - Static method in class net.snowflake.hivemetastoreconnector.commands.CreateExternalTable

Generate the string for a column to be used in the query

generateCredentialsString(String, Configuration) - Static method in class net.snowflake.hivemetastoreconnector.util.StageCredentialUtil

Get the credentials for the given stage in the url, for example: credentials=(AWS_KEY_ID='{accessKey}' AWS_SECRET_KEY='{awsSecretKey}')

generateSqlQueries() - Method in class net.snowflake.hivemetastoreconnector.commands.AddPartition

Generates the queries for add partition.

generateSqlQueries() - Method in class net.snowflake.hivemetastoreconnector.commands.AlterExternalTable

Generates the necessary queries on a Hive alter table event

generateSqlQueries() - Method in class net.snowflake.hivemetastoreconnector.commands.Command

Generates the query in a string form to be sent to Snowflake

generateSqlQueries() - Method in class net.snowflake.hivemetastoreconnector.commands.CreateExternalTable

Generates the queries for create external table The behavior of this method is as follows (in order of preference): a.

generateSqlQueries() - Method in class net.snowflake.hivemetastoreconnector.commands.DropExternalTable

Generates the necessary queries on a hive drop table event

generateSqlQueries() - Method in class net.snowflake.hivemetastoreconnector.commands.DropPartition

Generates the necessary commands on a Hive drop partition event

generateSqlQueries() - Method in class net.snowflake.hivemetastoreconnector.commands.LogCommand

Generates no-op logging queries, for example: SELECT NULL /* LOGS IN COMMENTS * /;

generateStageName(Table, SnowflakeConf) - Static method in class net.snowflake.hivemetastoreconnector.commands.CreateExternalTable

Helper method to generate a stage name for newly created stages.

get() - Method in interface net.snowflake.hivemetastoreconnector.core.SnowflakeClient.ThrowableSupplier

getCommand(ListenerEvent, SnowflakeConf) - Static method in class net.snowflake.hivemetastoreconnector.core.CommandGenerator

Creates a command based on the arguments Defers the actual creation to subclasses

getConnection(SnowflakeConf, String) - Static method in class net.snowflake.hivemetastoreconnector.core.SnowflakeClient

Get the connection to the Snowflake account.

getDatabaseName() - Method in class net.snowflake.hivemetastoreconnector.commands.Command

getSecret(String) - Method in class net.snowflake.hivemetastoreconnector.SnowflakeConf

Convenience method.

getSnowflakeDefaultSchema(SnowflakeConf) - Static method in class net.snowflake.hivemetastoreconnector.util.HiveToSnowflakeSchema

getSnowflakePropertyName() - Method in enum net.snowflake.hivemetastoreconnector.SnowflakeConf.ConfVars

getSnowflakeSchemaFromHiveSchema(String, String, Set<String>) - Static method in class net.snowflake.hivemetastoreconnector.util.HiveToSnowflakeSchema

getSnowflakeSchemaFromHiveSchema(String, SnowflakeConf) - Static method in class net.snowflake.hivemetastoreconnector.util.HiveToSnowflakeSchema

getSnowflakeSchemaSet(SnowflakeConf) - Static method in class net.snowflake.hivemetastoreconnector.util.HiveToSnowflakeSchema

getTableName() - Method in class net.snowflake.hivemetastoreconnector.commands.Command

getVarname() - Method in enum net.snowflake.hivemetastoreconnector.SnowflakeConf.ConfVars

H

HiveSyncTool - Class in net.snowflake.hivemetastoreconnector.core

Utility class for syncing metadata from Hive to Snowflake

HiveSyncTool(HiveMetaStoreClient) - Constructor for class net.snowflake.hivemetastoreconnector.core.HiveSyncTool

Instantiates the HiveSyncTool.

hiveToSnowflakeDataTypeMap - Static variable in class net.snowflake.hivemetastoreconnector.util.HiveToSnowflakeType

The mapping from a Hive datatype to a Snowflake datatype

HiveToSnowflakeSchema - Class in net.snowflake.hivemetastoreconnector.util

A util to get the correct Snowflake schema from the Hive schema.

HiveToSnowflakeSchema() - Constructor for class net.snowflake.hivemetastoreconnector.util.HiveToSnowflakeSchema

HiveToSnowflakeType - Class in net.snowflake.hivemetastoreconnector.util

A util to convert Hive types such as the hive datatype to Snowflake types.

HiveToSnowflakeType() - Constructor for class net.snowflake.hivemetastoreconnector.util.HiveToSnowflakeType

HiveToSnowflakeType.SnowflakeFileFormatType - Enum in net.snowflake.hivemetastoreconnector.util

The file format types suppported by Snowflake

hiveTypesWithSpecifications - Static variable in class net.snowflake.hivemetastoreconnector.util.HiveToSnowflakeType

The Hive data types with specifications, e.g.

I

isCompact() - Method in class net.snowflake.hivemetastoreconnector.commands.AddPartition

Mthod to determine whether a command was already compacted

isSnowflakeJDBCProperty() - Method in enum net.snowflake.hivemetastoreconnector.SnowflakeConf.ConfVars

J

JSON - net.snowflake.hivemetastoreconnector.util.HiveToSnowflakeType.SnowflakeFileFormatType

L

LogCommand - Class in net.snowflake.hivemetastoreconnector.commands

A class for no-op commands used for logging purposes, as queries sent to Snowflake will be logged.

LogCommand(Table, Exception) - Constructor for class net.snowflake.hivemetastoreconnector.commands.LogCommand

An overload of the constructor to log errors

LogCommand(Table, String) - Constructor for class net.snowflake.hivemetastoreconnector.commands.LogCommand

Constructor for LogCommand

M

main(String[]) - Static method in class net.snowflake.hivemetastoreconnector.core.HiveSyncTool

A convenient entry point for the sync tool.

N

net.snowflake.hivemetastoreconnector - package net.snowflake.hivemetastoreconnector

net.snowflake.hivemetastoreconnector.commands - package net.snowflake.hivemetastoreconnector.commands

net.snowflake.hivemetastoreconnector.core - package net.snowflake.hivemetastoreconnector.core

net.snowflake.hivemetastoreconnector.util - package net.snowflake.hivemetastoreconnector.util

O

onAddPartition(AddPartitionEvent) - Method in class net.snowflake.hivemetastoreconnector.SnowflakeHiveListener

The listener for the add partition command

onAlterPartition(AlterPartitionEvent) - Method in class net.snowflake.hivemetastoreconnector.SnowflakeHiveListener

The listener for the alter partition command

onAlterTable(AlterTableEvent) - Method in class net.snowflake.hivemetastoreconnector.SnowflakeHiveListener

The listener for the alter table command

onCreateTable(CreateTableEvent) - Method in class net.snowflake.hivemetastoreconnector.SnowflakeHiveListener

The listener for the create table command

onDropPartition(DropPartitionEvent) - Method in class net.snowflake.hivemetastoreconnector.SnowflakeHiveListener

The listener for the drop partition command

onDropTable(DropTableEvent) - Method in class net.snowflake.hivemetastoreconnector.SnowflakeHiveListener

The listener for the drop table command

ORC - net.snowflake.hivemetastoreconnector.util.HiveToSnowflakeType.SnowflakeFileFormatType

P

PARQUET - net.snowflake.hivemetastoreconnector.util.HiveToSnowflakeType.SnowflakeFileFormatType

R

relativizePartitionURI(Table, Partition) - Static method in class net.snowflake.hivemetastoreconnector.util.StringUtil

Helper method for getting a relative path between a Hive table and Hive partition.

relativizeURI(String, String) - Static method in class net.snowflake.hivemetastoreconnector.util.StringUtil

Helper method for getting a relative path between two URIs.

retry(SnowflakeClient.ThrowableSupplier<T, E>, SnowflakeConf) - Static method in class net.snowflake.hivemetastoreconnector.core.SnowflakeClient

Helper method for simple retries.

S

Scheduler - Class in net.snowflake.hivemetastoreconnector.core

Utility class that allows messages to be queued and processed in the background.

Scheduler(int, SnowflakeConf) - Constructor for class net.snowflake.hivemetastoreconnector.core.Scheduler

Constructor for the scheduler

SNOWFLAKE_CLIENT_FORCE_SYNCHRONOUS - net.snowflake.hivemetastoreconnector.SnowflakeConf.ConfVars

SNOWFLAKE_CLIENT_THREAD_COUNT - net.snowflake.hivemetastoreconnector.SnowflakeConf.ConfVars

SNOWFLAKE_DATA_COLUMN_CASING - net.snowflake.hivemetastoreconnector.SnowflakeConf.ConfVars

SNOWFLAKE_DATABASE_FILTER_REGEX - net.snowflake.hivemetastoreconnector.SnowflakeConf.ConfVars

SNOWFLAKE_ENABLE_CREDENTIALS_FROM_HIVE_CONF - net.snowflake.hivemetastoreconnector.SnowflakeConf.ConfVars

SNOWFLAKE_HIVEMETASTORELISTENER_RETRY_COUNT - net.snowflake.hivemetastoreconnector.SnowflakeConf.ConfVars

SNOWFLAKE_HIVEMETASTORELISTENER_RETRY_TIMEOUT_MILLISECONDS - net.snowflake.hivemetastoreconnector.SnowflakeConf.ConfVars

SNOWFLAKE_INTEGRATION_FOR_HIVE_EXTERNAL_TABLES - net.snowflake.hivemetastoreconnector.SnowflakeConf.ConfVars

SNOWFLAKE_JDBC_ACCOUNT - net.snowflake.hivemetastoreconnector.SnowflakeConf.ConfVars

SNOWFLAKE_JDBC_CONNECTION - net.snowflake.hivemetastoreconnector.SnowflakeConf.ConfVars

SNOWFLAKE_JDBC_DB - net.snowflake.hivemetastoreconnector.SnowflakeConf.ConfVars

SNOWFLAKE_JDBC_PASSWORD - net.snowflake.hivemetastoreconnector.SnowflakeConf.ConfVars

SNOWFLAKE_JDBC_PRIVATE_KEY - net.snowflake.hivemetastoreconnector.SnowflakeConf.ConfVars

SNOWFLAKE_JDBC_ROLE - net.snowflake.hivemetastoreconnector.SnowflakeConf.ConfVars

SNOWFLAKE_JDBC_SCHEMA - net.snowflake.hivemetastoreconnector.SnowflakeConf.ConfVars

SNOWFLAKE_JDBC_SSL - net.snowflake.hivemetastoreconnector.SnowflakeConf.ConfVars

SNOWFLAKE_JDBC_USERNAME - net.snowflake.hivemetastoreconnector.SnowflakeConf.ConfVars

SNOWFLAKE_JDBC_WAREHOUSE - net.snowflake.hivemetastoreconnector.SnowflakeConf.ConfVars

SNOWFLAKE_SCHEMA_LIST - net.snowflake.hivemetastoreconnector.SnowflakeConf.ConfVars

SNOWFLAKE_STAGE_FOR_HIVE_EXTERNAL_TABLES - net.snowflake.hivemetastoreconnector.SnowflakeConf.ConfVars

SNOWFLAKE_TABLE_FILTER_REGEX - net.snowflake.hivemetastoreconnector.SnowflakeConf.ConfVars

SnowflakeClient - Class in net.snowflake.hivemetastoreconnector.core

Class that uses the snowflake jdbc to connect to snowflake.

SnowflakeClient() - Constructor for class net.snowflake.hivemetastoreconnector.core.SnowflakeClient

SnowflakeClient.ThrowableSupplier<T,E extends Throwable> - Interface in net.snowflake.hivemetastoreconnector.core

Helper interface that represents a Supplier that can throw an exception.

SnowflakeConf - Class in net.snowflake.hivemetastoreconnector

The configuration for the Snowflake Hive metastore listener, which is stored in snowflake-config.xml

SnowflakeConf() - Constructor for class net.snowflake.hivemetastoreconnector.SnowflakeConf

SnowflakeConf.ConfVars - Enum in net.snowflake.hivemetastoreconnector

SnowflakeHiveListener - Class in net.snowflake.hivemetastoreconnector

The listener for Hive This listener will get notified when the corresponding command is called

SnowflakeHiveListener(Configuration) - Constructor for class net.snowflake.hivemetastoreconnector.SnowflakeHiveListener

StageCredentialUtil - Class in net.snowflake.hivemetastoreconnector.util

A utility function to help with retrieving stage credentials and generating the credential string for Snowflake.

StageCredentialUtil() - Constructor for class net.snowflake.hivemetastoreconnector.util.StageCredentialUtil

StageCredentialUtil.StageCredentialType - Enum in net.snowflake.hivemetastoreconnector.util

StringUtil - Class in net.snowflake.hivemetastoreconnector.util

A utility class for formatting strings

StringUtil() - Constructor for class net.snowflake.hivemetastoreconnector.util.StringUtil

sync() - Method in class net.snowflake.hivemetastoreconnector.core.HiveSyncTool

Does a one-time, one-way metadata sync from the Hive metastore to Snowflake.

T

toSnowflakeColumnDataType(String) - Static method in class net.snowflake.hivemetastoreconnector.util.HiveToSnowflakeType

converts a Hive column data type to a Snowflake datatype

toSnowflakeFileFormat(HiveToSnowflakeType.SnowflakeFileFormatType, SerDeInfo, Map<String, String>) - Static method in class net.snowflake.hivemetastoreconnector.util.HiveToSnowflakeType

converts a Hive file format to a Snowflake file format

toSnowflakeFileFormatType(String, String) - Static method in class net.snowflake.hivemetastoreconnector.util.HiveToSnowflakeType

Determines the most appropriate Snowflake file format type for a given Hive file format and SerDe.

toSnowflakeURL(String) - Static method in class net.snowflake.hivemetastoreconnector.util.HiveToSnowflakeType

Converts a Hive URL to a Snowflake URL.

V

valueOf(String) - Static method in enum net.snowflake.hivemetastoreconnector.SnowflakeConf.ConfVars

Returns the enum constant of this type with the specified name.

valueOf(String) - Static method in enum net.snowflake.hivemetastoreconnector.util.HiveToSnowflakeType.SnowflakeFileFormatType

Returns the enum constant of this type with the specified name.

valueOf(String) - Static method in enum net.snowflake.hivemetastoreconnector.util.StageCredentialUtil.StageCredentialType

Returns the enum constant of this type with the specified name.

values() - Static method in enum net.snowflake.hivemetastoreconnector.SnowflakeConf.ConfVars

Returns an array containing the constants of this enum type, in the order they are declared.

values() - Static method in enum net.snowflake.hivemetastoreconnector.util.HiveToSnowflakeType.SnowflakeFileFormatType

Returns an array containing the constants of this enum type, in the order they are declared.

values() - Static method in enum net.snowflake.hivemetastoreconnector.util.StageCredentialUtil.StageCredentialType

Returns an array containing the constants of this enum type, in the order they are declared.

X

XML - net.snowflake.hivemetastoreconnector.util.HiveToSnowflakeType.SnowflakeFileFormatType

A B C D E F G H I J L M N O P R S T V X
All Classes All Packages