Class OrcSerDe
- java.lang.Object
-
- software.amazon.awssdk.services.firehose.model.OrcSerDe
-
- All Implemented Interfaces:
Serializable,SdkPojo,ToCopyableBuilder<OrcSerDe.Builder,OrcSerDe>
@Generated("software.amazon.awssdk:codegen") public final class OrcSerDe extends Object implements SdkPojo, Serializable, ToCopyableBuilder<OrcSerDe.Builder,OrcSerDe>
A serializer to use for converting data to the ORC format before storing it in Amazon S3. For more information, see Apache ORC.
- See Also:
- Serialized Form
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static interfaceOrcSerDe.Builder
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description IntegerblockSizeBytes()The Hadoop Distributed File System (HDFS) block size.List<String>bloomFilterColumns()The column names for which you want Kinesis Data Firehose to create bloom filters.DoublebloomFilterFalsePositiveProbability()The Bloom filter false positive probability (FPP).static OrcSerDe.Builderbuilder()OrcCompressioncompression()The compression code to use over data blocks.StringcompressionAsString()The compression code to use over data blocks.DoubledictionaryKeyThreshold()Represents the fraction of the total number of non-null rows.BooleanenablePadding()Set this totrueto indicate that you want stripes to be padded to the HDFS block boundaries.booleanequals(Object obj)booleanequalsBySdkFields(Object obj)OrcFormatVersionformatVersion()The version of the file to write.StringformatVersionAsString()The version of the file to write.<T> Optional<T>getValueForField(String fieldName, Class<T> clazz)booleanhasBloomFilterColumns()For responses, this returns true if the service returned a value for the BloomFilterColumns property.inthashCode()DoublepaddingTolerance()A number between 0 and 1 that defines the tolerance for block padding as a decimal fraction of stripe size.IntegerrowIndexStride()The number of rows between index entries.List<SdkField<?>>sdkFields()static Class<? extends OrcSerDe.Builder>serializableBuilderClass()IntegerstripeSizeBytes()The number of bytes in each stripe.OrcSerDe.BuildertoBuilder()StringtoString()Returns a string representation of this object.-
Methods inherited from class java.lang.Object
clone, finalize, getClass, notify, notifyAll, wait, wait, wait
-
Methods inherited from interface software.amazon.awssdk.utils.builder.ToCopyableBuilder
copy
-
-
-
-
Method Detail
-
stripeSizeBytes
public final Integer stripeSizeBytes()
The number of bytes in each stripe. The default is 64 MiB and the minimum is 8 MiB.
- Returns:
- The number of bytes in each stripe. The default is 64 MiB and the minimum is 8 MiB.
-
blockSizeBytes
public final Integer blockSizeBytes()
The Hadoop Distributed File System (HDFS) block size. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 256 MiB and the minimum is 64 MiB. Kinesis Data Firehose uses this value for padding calculations.
- Returns:
- The Hadoop Distributed File System (HDFS) block size. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 256 MiB and the minimum is 64 MiB. Kinesis Data Firehose uses this value for padding calculations.
-
rowIndexStride
public final Integer rowIndexStride()
The number of rows between index entries. The default is 10,000 and the minimum is 1,000.
- Returns:
- The number of rows between index entries. The default is 10,000 and the minimum is 1,000.
-
enablePadding
public final Boolean enablePadding()
Set this to
trueto indicate that you want stripes to be padded to the HDFS block boundaries. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default isfalse.- Returns:
- Set this to
trueto indicate that you want stripes to be padded to the HDFS block boundaries. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default isfalse.
-
paddingTolerance
public final Double paddingTolerance()
A number between 0 and 1 that defines the tolerance for block padding as a decimal fraction of stripe size. The default value is 0.05, which means 5 percent of stripe size.
For the default values of 64 MiB ORC stripes and 256 MiB HDFS blocks, the default block padding tolerance of 5 percent reserves a maximum of 3.2 MiB for padding within the 256 MiB block. In such a case, if the available size within the block is more than 3.2 MiB, a new, smaller stripe is inserted to fit within that space. This ensures that no stripe crosses block boundaries and causes remote reads within a node-local task.
Kinesis Data Firehose ignores this parameter when OrcSerDe$EnablePadding is
false.- Returns:
- A number between 0 and 1 that defines the tolerance for block padding as a decimal fraction of stripe
size. The default value is 0.05, which means 5 percent of stripe size.
For the default values of 64 MiB ORC stripes and 256 MiB HDFS blocks, the default block padding tolerance of 5 percent reserves a maximum of 3.2 MiB for padding within the 256 MiB block. In such a case, if the available size within the block is more than 3.2 MiB, a new, smaller stripe is inserted to fit within that space. This ensures that no stripe crosses block boundaries and causes remote reads within a node-local task.
Kinesis Data Firehose ignores this parameter when OrcSerDe$EnablePadding is
false.
-
compression
public final OrcCompression compression()
The compression code to use over data blocks. The default is
SNAPPY.If the service returns an enum value that is not available in the current SDK version,
compressionwill returnOrcCompression.UNKNOWN_TO_SDK_VERSION. The raw value returned by the service is available fromcompressionAsString().- Returns:
- The compression code to use over data blocks. The default is
SNAPPY. - See Also:
OrcCompression
-
compressionAsString
public final String compressionAsString()
The compression code to use over data blocks. The default is
SNAPPY.If the service returns an enum value that is not available in the current SDK version,
compressionwill returnOrcCompression.UNKNOWN_TO_SDK_VERSION. The raw value returned by the service is available fromcompressionAsString().- Returns:
- The compression code to use over data blocks. The default is
SNAPPY. - See Also:
OrcCompression
-
hasBloomFilterColumns
public final boolean hasBloomFilterColumns()
For responses, this returns true if the service returned a value for the BloomFilterColumns property. This DOES NOT check that the value is non-empty (for which, you should check theisEmpty()method on the property). This is useful because the SDK will never return a null collection or map, but you may need to differentiate between the service returning nothing (or null) and the service returning an empty collection or map. For requests, this returns true if a value for the property was specified in the request builder, and false if a value was not specified.
-
bloomFilterColumns
public final List<String> bloomFilterColumns()
The column names for which you want Kinesis Data Firehose to create bloom filters. The default is
null.Attempts to modify the collection returned by this method will result in an UnsupportedOperationException.
This method will never return null. If you would like to know whether the service returned this field (so that you can differentiate between null and empty), you can use the
hasBloomFilterColumns()method.- Returns:
- The column names for which you want Kinesis Data Firehose to create bloom filters. The default is
null.
-
bloomFilterFalsePositiveProbability
public final Double bloomFilterFalsePositiveProbability()
The Bloom filter false positive probability (FPP). The lower the FPP, the bigger the Bloom filter. The default value is 0.05, the minimum is 0, and the maximum is 1.
- Returns:
- The Bloom filter false positive probability (FPP). The lower the FPP, the bigger the Bloom filter. The default value is 0.05, the minimum is 0, and the maximum is 1.
-
dictionaryKeyThreshold
public final Double dictionaryKeyThreshold()
Represents the fraction of the total number of non-null rows. To turn off dictionary encoding, set this fraction to a number that is less than the number of distinct keys in a dictionary. To always use dictionary encoding, set this threshold to 1.
- Returns:
- Represents the fraction of the total number of non-null rows. To turn off dictionary encoding, set this fraction to a number that is less than the number of distinct keys in a dictionary. To always use dictionary encoding, set this threshold to 1.
-
formatVersion
public final OrcFormatVersion formatVersion()
The version of the file to write. The possible values are
V0_11andV0_12. The default isV0_12.If the service returns an enum value that is not available in the current SDK version,
formatVersionwill returnOrcFormatVersion.UNKNOWN_TO_SDK_VERSION. The raw value returned by the service is available fromformatVersionAsString().- Returns:
- The version of the file to write. The possible values are
V0_11andV0_12. The default isV0_12. - See Also:
OrcFormatVersion
-
formatVersionAsString
public final String formatVersionAsString()
The version of the file to write. The possible values are
V0_11andV0_12. The default isV0_12.If the service returns an enum value that is not available in the current SDK version,
formatVersionwill returnOrcFormatVersion.UNKNOWN_TO_SDK_VERSION. The raw value returned by the service is available fromformatVersionAsString().- Returns:
- The version of the file to write. The possible values are
V0_11andV0_12. The default isV0_12. - See Also:
OrcFormatVersion
-
toBuilder
public OrcSerDe.Builder toBuilder()
- Specified by:
toBuilderin interfaceToCopyableBuilder<OrcSerDe.Builder,OrcSerDe>
-
builder
public static OrcSerDe.Builder builder()
-
serializableBuilderClass
public static Class<? extends OrcSerDe.Builder> serializableBuilderClass()
-
equalsBySdkFields
public final boolean equalsBySdkFields(Object obj)
- Specified by:
equalsBySdkFieldsin interfaceSdkPojo
-
toString
public final String toString()
Returns a string representation of this object. This is useful for testing and debugging. Sensitive data will be redacted from this string using a placeholder value.
-
-