org.apache.parquet.crypto.keytools

Class KeyToolkit

java.lang.Object

org.apache.parquet.crypto.keytools.KeyToolkit

public class KeyToolkit
extends Object

Field Summary

Fields

Modifier and Type	Field and Description
static long	CACHE_LIFETIME_DEFAULT_SECONDS
static String	CACHE_LIFETIME_PROPERTY_NAME Lifetime of cached entities (key encryption keys, local wrapping keys, KMS client objects).
static int	DATA_KEY_LENGTH_DEFAULT
static String	DATA_KEY_LENGTH_PROPERTY_NAME Length of data encryption keys (DEKs), randomly generated by parquet key management tools.
static boolean	DOUBLE_WRAPPING_DEFAULT
static String	DOUBLE_WRAPPING_PROPERTY_NAME Use double wrapping - where data encryption keys (DEKs) are encrypted with key encryption keys (KEKs), which in turn are encrypted with master keys.
static int	KEK_LENGTH_DEFAULT
static String	KEK_LENGTH_PROPERTY_NAME Length of key encryption keys (KEKs), randomly generated by parquet key management tools.
static String	KEY_ACCESS_TOKEN_PROPERTY_NAME Authorization token that will be passed to KMS.
static boolean	KEY_MATERIAL_INTERNAL_DEFAULT
static String	KEY_MATERIAL_INTERNAL_PROPERTY_NAME Store key material inside Parquet file footers; this mode doesn’t produce additional files.
static String	KMS_CLIENT_CLASS_PROPERTY_NAME Class implementing the KmsClient interface.
static String	KMS_INSTANCE_ID_PROPERTY_NAME ID of the KMS instance that will be used for encryption (if multiple KMS instances are available).
static String	KMS_INSTANCE_URL_PROPERTY_NAME URL of the KMS instance.

Constructor Summary

Constructors

Constructor and Description
KeyToolkit()

Method Summary

Modifier and Type	Method and Description
static byte[]	decryptKeyLocally(String encodedEncryptedKey, byte[] masterKeyBytes, byte[] AAD) Decrypts encrypted key with "masterKey", using AES-GCM and the "AAD"
static String	encryptKeyLocally(byte[] keyBytes, byte[] masterKeyBytes, byte[] AAD) Encrypts "key" with "masterKey", using AES-GCM and the "AAD"
static void	removeCacheEntriesForAllTokens()
static void	removeCacheEntriesForToken(String accessToken) Flush any caches that are tied to the (compromised) accessToken
static void	rotateMasterKeys(String folderPath, org.apache.hadoop.conf.Configuration hadoopConfig) Key rotation.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

KMS_CLIENT_CLASS_PROPERTY_NAME

public static final String KMS_CLIENT_CLASS_PROPERTY_NAME

Class implementing the KmsClient interface. KMS stands for “key management service”.

See Also:

Constant Field Values

KMS_INSTANCE_ID_PROPERTY_NAME

public static final String KMS_INSTANCE_ID_PROPERTY_NAME

ID of the KMS instance that will be used for encryption (if multiple KMS instances are available).

See Also:

Constant Field Values

KMS_INSTANCE_URL_PROPERTY_NAME

public static final String KMS_INSTANCE_URL_PROPERTY_NAME

URL of the KMS instance.

See Also:

Constant Field Values

KEY_ACCESS_TOKEN_PROPERTY_NAME

public static final String KEY_ACCESS_TOKEN_PROPERTY_NAME

Authorization token that will be passed to KMS.

See Also:

Constant Field Values

DOUBLE_WRAPPING_PROPERTY_NAME

public static final String DOUBLE_WRAPPING_PROPERTY_NAME

Use double wrapping - where data encryption keys (DEKs) are encrypted with key encryption keys (KEKs), which in turn are encrypted with master keys. By default, true. If set to false, DEKs are directly encrypted with master keys, KEKs are not used.

See Also:

Constant Field Values

CACHE_LIFETIME_PROPERTY_NAME

public static final String CACHE_LIFETIME_PROPERTY_NAME

Lifetime of cached entities (key encryption keys, local wrapping keys, KMS client objects).

See Also:

Constant Field Values

KEY_MATERIAL_INTERNAL_PROPERTY_NAME

public static final String KEY_MATERIAL_INTERNAL_PROPERTY_NAME

Store key material inside Parquet file footers; this mode doesn’t produce additional files. By default, true. If set to false, key material is stored in separate files in the same folder, which enables key rotation for immutable Parquet files.

See Also:

Constant Field Values

DATA_KEY_LENGTH_PROPERTY_NAME

public static final String DATA_KEY_LENGTH_PROPERTY_NAME

Length of data encryption keys (DEKs), randomly generated by parquet key management tools. Can be 128, 192 or 256 bits.

See Also:

Constant Field Values

KEK_LENGTH_PROPERTY_NAME

public static final String KEK_LENGTH_PROPERTY_NAME

Length of key encryption keys (KEKs), randomly generated by parquet key management tools. Can be 128, 192 or 256 bits.

See Also:

Constant Field Values

DOUBLE_WRAPPING_DEFAULT

public static final boolean DOUBLE_WRAPPING_DEFAULT

See Also:

Constant Field Values

CACHE_LIFETIME_DEFAULT_SECONDS

public static final long CACHE_LIFETIME_DEFAULT_SECONDS

See Also:

Constant Field Values

KEY_MATERIAL_INTERNAL_DEFAULT

public static final boolean KEY_MATERIAL_INTERNAL_DEFAULT

See Also:

Constant Field Values

DATA_KEY_LENGTH_DEFAULT

public static final int DATA_KEY_LENGTH_DEFAULT

See Also:

Constant Field Values

KEK_LENGTH_DEFAULT

public static final int KEK_LENGTH_DEFAULT

See Also:

Constant Field Values

Constructor Detail

KeyToolkit

public KeyToolkit()

Method Detail

rotateMasterKeys

public static void rotateMasterKeys(String folderPath,
                                    org.apache.hadoop.conf.Configuration hadoopConfig)
                             throws IOException,
                                    ParquetCryptoRuntimeException,
                                    KeyAccessDeniedException,
                                    UnsupportedOperationException

Key rotation. In the single wrapping mode, decrypts data keys with old master keys, then encrypts them with new master keys. In the double wrapping mode, decrypts KEKs (key encryption keys) with old master keys, generates new KEKs and encrypts them with new master keys. Works only if key material is not stored internally in file footers. Not supported in local key wrapping mode. Method can be run by multiple threads, but each thread must work on a different folder.

Parameters:

folderPath - parent path of Parquet files, whose keys will be rotated

hadoopConfig - Hadoop configuration

Throws:

IOException - I/O problems

ParquetCryptoRuntimeException - General parquet encryption problems

KeyAccessDeniedException - No access to master keys

UnsupportedOperationException - Master key rotation not supported in the specific configuration

removeCacheEntriesForToken

public static void removeCacheEntriesForToken(String accessToken)

Flush any caches that are tied to the (compromised) accessToken

Parameters:

accessToken - access token

removeCacheEntriesForAllTokens

public static void removeCacheEntriesForAllTokens()

encryptKeyLocally

public static String encryptKeyLocally(byte[] keyBytes,
                                       byte[] masterKeyBytes,
                                       byte[] AAD)

Encrypts "key" with "masterKey", using AES-GCM and the "AAD"

Parameters:

keyBytes - the key to encrypt

masterKeyBytes - encryption key

AAD - additional authenticated data

Returns:

base64 encoded encrypted key

decryptKeyLocally

public static byte[] decryptKeyLocally(String encodedEncryptedKey,
                                       byte[] masterKeyBytes,
                                       byte[] AAD)

Decrypts encrypted key with "masterKey", using AES-GCM and the "AAD"

Parameters:

encodedEncryptedKey - base64 encoded encrypted key

masterKeyBytes - encryption key

AAD - additional authenticated data

Returns:

decrypted key