Class S3CsvSource
- java.lang.Object
-
- software.amazon.awssdk.services.glue.model.S3CsvSource
-
- All Implemented Interfaces:
Serializable,SdkPojo,ToCopyableBuilder<S3CsvSource.Builder,S3CsvSource>
@Generated("software.amazon.awssdk:codegen") public final class S3CsvSource extends Object implements SdkPojo, Serializable, ToCopyableBuilder<S3CsvSource.Builder,S3CsvSource>
Specifies a command-separated value (CSV) data store stored in Amazon S3.
- See Also:
- Serialized Form
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static interfaceS3CsvSource.Builder
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description S3DirectSourceAdditionalOptionsadditionalOptions()Specifies additional connection options.static S3CsvSource.Builderbuilder()CompressionTypecompressionType()Specifies how the data is compressed.StringcompressionTypeAsString()Specifies how the data is compressed.booleanequals(Object obj)booleanequalsBySdkFields(Object obj)Stringescaper()Specifies a character to use for escaping.List<String>exclusions()A string containing a JSON list of Unix-style glob patterns to exclude.<T> Optional<T>getValueForField(String fieldName, Class<T> clazz)StringgroupFiles()Grouping files is turned on by default when the input contains more than 50,000 files.StringgroupSize()The target group size in bytes.booleanhasExclusions()For responses, this returns true if the service returned a value for the Exclusions property.inthashCode()booleanhasOutputSchemas()For responses, this returns true if the service returned a value for the OutputSchemas property.booleanhasPaths()For responses, this returns true if the service returned a value for the Paths property.IntegermaxBand()This option controls the duration in milliseconds after which the s3 listing is likely to be consistent.IntegermaxFilesInBand()This option specifies the maximum number of files to save from the last maxBand seconds.Booleanmultiline()A Boolean value that specifies whether a single record can span multiple lines.Stringname()The name of the data store.BooleanoptimizePerformance()A Boolean value that specifies whether to use the advanced SIMD CSV reader along with Apache Arrow based columnar memory formats.List<GlueSchema>outputSchemas()Specifies the data schema for the S3 CSV source.List<String>paths()A list of the Amazon S3 paths to read from.QuoteCharquoteChar()Specifies the character to use for quoting.StringquoteCharAsString()Specifies the character to use for quoting.Booleanrecurse()If set to true, recursively reads files in all subdirectories under the specified paths.Map<String,SdkField<?>>sdkFieldNameToField()List<SdkField<?>>sdkFields()Separatorseparator()Specifies the delimiter character.StringseparatorAsString()Specifies the delimiter character.static Class<? extends S3CsvSource.Builder>serializableBuilderClass()BooleanskipFirst()A Boolean value that specifies whether to skip the first data line.S3CsvSource.BuildertoBuilder()StringtoString()Returns a string representation of this object.BooleanwithHeader()A Boolean value that specifies whether to treat the first line as a header.BooleanwriteHeader()A Boolean value that specifies whether to write the header to output.-
Methods inherited from class java.lang.Object
clone, finalize, getClass, notify, notifyAll, wait, wait, wait
-
Methods inherited from interface software.amazon.awssdk.utils.builder.ToCopyableBuilder
copy
-
-
-
-
Method Detail
-
name
public final String name()
The name of the data store.
- Returns:
- The name of the data store.
-
hasPaths
public final boolean hasPaths()
For responses, this returns true if the service returned a value for the Paths property. This DOES NOT check that the value is non-empty (for which, you should check theisEmpty()method on the property). This is useful because the SDK will never return a null collection or map, but you may need to differentiate between the service returning nothing (or null) and the service returning an empty collection or map. For requests, this returns true if a value for the property was specified in the request builder, and false if a value was not specified.
-
paths
public final List<String> paths()
A list of the Amazon S3 paths to read from.
Attempts to modify the collection returned by this method will result in an UnsupportedOperationException.
This method will never return null. If you would like to know whether the service returned this field (so that you can differentiate between null and empty), you can use the
hasPaths()method.- Returns:
- A list of the Amazon S3 paths to read from.
-
compressionType
public final CompressionType compressionType()
Specifies how the data is compressed. This is generally not necessary if the data has a standard file extension. Possible values are
"gzip"and"bzip").If the service returns an enum value that is not available in the current SDK version,
compressionTypewill returnCompressionType.UNKNOWN_TO_SDK_VERSION. The raw value returned by the service is available fromcompressionTypeAsString().- Returns:
- Specifies how the data is compressed. This is generally not necessary if the data has a standard file
extension. Possible values are
"gzip"and"bzip"). - See Also:
CompressionType
-
compressionTypeAsString
public final String compressionTypeAsString()
Specifies how the data is compressed. This is generally not necessary if the data has a standard file extension. Possible values are
"gzip"and"bzip").If the service returns an enum value that is not available in the current SDK version,
compressionTypewill returnCompressionType.UNKNOWN_TO_SDK_VERSION. The raw value returned by the service is available fromcompressionTypeAsString().- Returns:
- Specifies how the data is compressed. This is generally not necessary if the data has a standard file
extension. Possible values are
"gzip"and"bzip"). - See Also:
CompressionType
-
hasExclusions
public final boolean hasExclusions()
For responses, this returns true if the service returned a value for the Exclusions property. This DOES NOT check that the value is non-empty (for which, you should check theisEmpty()method on the property). This is useful because the SDK will never return a null collection or map, but you may need to differentiate between the service returning nothing (or null) and the service returning an empty collection or map. For requests, this returns true if a value for the property was specified in the request builder, and false if a value was not specified.
-
exclusions
public final List<String> exclusions()
A string containing a JSON list of Unix-style glob patterns to exclude. For example, "[\"**.pdf\"]" excludes all PDF files.
Attempts to modify the collection returned by this method will result in an UnsupportedOperationException.
This method will never return null. If you would like to know whether the service returned this field (so that you can differentiate between null and empty), you can use the
hasExclusions()method.- Returns:
- A string containing a JSON list of Unix-style glob patterns to exclude. For example, "[\"**.pdf\"]" excludes all PDF files.
-
groupSize
public final String groupSize()
The target group size in bytes. The default is computed based on the input data size and the size of your cluster. When there are fewer than 50,000 input files,
"groupFiles"must be set to"inPartition"for this to take effect.- Returns:
- The target group size in bytes. The default is computed based on the input data size and the size of your
cluster. When there are fewer than 50,000 input files,
"groupFiles"must be set to"inPartition"for this to take effect.
-
groupFiles
public final String groupFiles()
Grouping files is turned on by default when the input contains more than 50,000 files. To turn on grouping with fewer than 50,000 files, set this parameter to "inPartition". To disable grouping when there are more than 50,000 files, set this parameter to
"none".- Returns:
- Grouping files is turned on by default when the input contains more than 50,000 files. To turn on
grouping with fewer than 50,000 files, set this parameter to "inPartition". To disable grouping when
there are more than 50,000 files, set this parameter to
"none".
-
recurse
public final Boolean recurse()
If set to true, recursively reads files in all subdirectories under the specified paths.
- Returns:
- If set to true, recursively reads files in all subdirectories under the specified paths.
-
maxBand
public final Integer maxBand()
This option controls the duration in milliseconds after which the s3 listing is likely to be consistent. Files with modification timestamps falling within the last maxBand milliseconds are tracked specially when using JobBookmarks to account for Amazon S3 eventual consistency. Most users don't need to set this option. The default is 900000 milliseconds, or 15 minutes.
- Returns:
- This option controls the duration in milliseconds after which the s3 listing is likely to be consistent. Files with modification timestamps falling within the last maxBand milliseconds are tracked specially when using JobBookmarks to account for Amazon S3 eventual consistency. Most users don't need to set this option. The default is 900000 milliseconds, or 15 minutes.
-
maxFilesInBand
public final Integer maxFilesInBand()
This option specifies the maximum number of files to save from the last maxBand seconds. If this number is exceeded, extra files are skipped and only processed in the next job run.
- Returns:
- This option specifies the maximum number of files to save from the last maxBand seconds. If this number is exceeded, extra files are skipped and only processed in the next job run.
-
additionalOptions
public final S3DirectSourceAdditionalOptions additionalOptions()
Specifies additional connection options.
- Returns:
- Specifies additional connection options.
-
separator
public final Separator separator()
Specifies the delimiter character. The default is a comma: ",", but any other character can be specified.
If the service returns an enum value that is not available in the current SDK version,
separatorwill returnSeparator.UNKNOWN_TO_SDK_VERSION. The raw value returned by the service is available fromseparatorAsString().- Returns:
- Specifies the delimiter character. The default is a comma: ",", but any other character can be specified.
- See Also:
Separator
-
separatorAsString
public final String separatorAsString()
Specifies the delimiter character. The default is a comma: ",", but any other character can be specified.
If the service returns an enum value that is not available in the current SDK version,
separatorwill returnSeparator.UNKNOWN_TO_SDK_VERSION. The raw value returned by the service is available fromseparatorAsString().- Returns:
- Specifies the delimiter character. The default is a comma: ",", but any other character can be specified.
- See Also:
Separator
-
escaper
public final String escaper()
Specifies a character to use for escaping. This option is used only when reading CSV files. The default value is
none. If enabled, the character which immediately follows is used as-is, except for a small set of well-known escapes (\n,\r,\t, and\0).- Returns:
- Specifies a character to use for escaping. This option is used only when reading CSV files. The default
value is
none. If enabled, the character which immediately follows is used as-is, except for a small set of well-known escapes (\n,\r,\t, and\0).
-
quoteChar
public final QuoteChar quoteChar()
Specifies the character to use for quoting. The default is a double quote:
'"'. Set this to-1to turn off quoting entirely.If the service returns an enum value that is not available in the current SDK version,
quoteCharwill returnQuoteChar.UNKNOWN_TO_SDK_VERSION. The raw value returned by the service is available fromquoteCharAsString().- Returns:
- Specifies the character to use for quoting. The default is a double quote:
'"'. Set this to-1to turn off quoting entirely. - See Also:
QuoteChar
-
quoteCharAsString
public final String quoteCharAsString()
Specifies the character to use for quoting. The default is a double quote:
'"'. Set this to-1to turn off quoting entirely.If the service returns an enum value that is not available in the current SDK version,
quoteCharwill returnQuoteChar.UNKNOWN_TO_SDK_VERSION. The raw value returned by the service is available fromquoteCharAsString().- Returns:
- Specifies the character to use for quoting. The default is a double quote:
'"'. Set this to-1to turn off quoting entirely. - See Also:
QuoteChar
-
multiline
public final Boolean multiline()
A Boolean value that specifies whether a single record can span multiple lines. This can occur when a field contains a quoted new-line character. You must set this option to True if any record spans multiple lines. The default value is
False, which allows for more aggressive file-splitting during parsing.- Returns:
- A Boolean value that specifies whether a single record can span multiple lines. This can occur when a
field contains a quoted new-line character. You must set this option to True if any record spans multiple
lines. The default value is
False, which allows for more aggressive file-splitting during parsing.
-
withHeader
public final Boolean withHeader()
A Boolean value that specifies whether to treat the first line as a header. The default value is
False.- Returns:
- A Boolean value that specifies whether to treat the first line as a header. The default value is
False.
-
writeHeader
public final Boolean writeHeader()
A Boolean value that specifies whether to write the header to output. The default value is
True.- Returns:
- A Boolean value that specifies whether to write the header to output. The default value is
True.
-
skipFirst
public final Boolean skipFirst()
A Boolean value that specifies whether to skip the first data line. The default value is
False.- Returns:
- A Boolean value that specifies whether to skip the first data line. The default value is
False.
-
optimizePerformance
public final Boolean optimizePerformance()
A Boolean value that specifies whether to use the advanced SIMD CSV reader along with Apache Arrow based columnar memory formats. Only available in Glue version 3.0.
- Returns:
- A Boolean value that specifies whether to use the advanced SIMD CSV reader along with Apache Arrow based columnar memory formats. Only available in Glue version 3.0.
-
hasOutputSchemas
public final boolean hasOutputSchemas()
For responses, this returns true if the service returned a value for the OutputSchemas property. This DOES NOT check that the value is non-empty (for which, you should check theisEmpty()method on the property). This is useful because the SDK will never return a null collection or map, but you may need to differentiate between the service returning nothing (or null) and the service returning an empty collection or map. For requests, this returns true if a value for the property was specified in the request builder, and false if a value was not specified.
-
outputSchemas
public final List<GlueSchema> outputSchemas()
Specifies the data schema for the S3 CSV source.
Attempts to modify the collection returned by this method will result in an UnsupportedOperationException.
This method will never return null. If you would like to know whether the service returned this field (so that you can differentiate between null and empty), you can use the
hasOutputSchemas()method.- Returns:
- Specifies the data schema for the S3 CSV source.
-
toBuilder
public S3CsvSource.Builder toBuilder()
- Specified by:
toBuilderin interfaceToCopyableBuilder<S3CsvSource.Builder,S3CsvSource>
-
builder
public static S3CsvSource.Builder builder()
-
serializableBuilderClass
public static Class<? extends S3CsvSource.Builder> serializableBuilderClass()
-
equalsBySdkFields
public final boolean equalsBySdkFields(Object obj)
- Specified by:
equalsBySdkFieldsin interfaceSdkPojo
-
toString
public final String toString()
Returns a string representation of this object. This is useful for testing and debugging. Sensitive data will be redacted from this string using a placeholder value.
-
sdkFieldNameToField
public final Map<String,SdkField<?>> sdkFieldNameToField()
- Specified by:
sdkFieldNameToFieldin interfaceSdkPojo
-
-