software.amazon.awssdk.services.glue.model

Interface S3CsvSource.Builder

All Superinterfaces:

Buildable, CopyableBuilder<S3CsvSource.Builder,S3CsvSource>, SdkBuilder<S3CsvSource.Builder,S3CsvSource>, SdkPojo

Enclosing class:

S3CsvSource

public static interface S3CsvSource.Builder
extends SdkPojo, CopyableBuilder<S3CsvSource.Builder,S3CsvSource>

Method Summary

Modifier and Type	Method and Description
default S3CsvSource.Builder	additionalOptions(Consumer<S3DirectSourceAdditionalOptions.Builder> additionalOptions) Specifies additional connection options.
S3CsvSource.Builder	additionalOptions(S3DirectSourceAdditionalOptions additionalOptions) Specifies additional connection options.
S3CsvSource.Builder	compressionType(CompressionType compressionType) Specifies how the data is compressed.
S3CsvSource.Builder	compressionType(String compressionType) Specifies how the data is compressed.
S3CsvSource.Builder	escaper(String escaper) Specifies a character to use for escaping.
S3CsvSource.Builder	exclusions(Collection<String> exclusions) A string containing a JSON list of Unix-style glob patterns to exclude.
S3CsvSource.Builder	exclusions(String... exclusions) A string containing a JSON list of Unix-style glob patterns to exclude.
S3CsvSource.Builder	groupFiles(String groupFiles) Grouping files is turned on by default when the input contains more than 50,000 files.
S3CsvSource.Builder	groupSize(String groupSize) The target group size in bytes.
S3CsvSource.Builder	maxBand(Integer maxBand) This option controls the duration in milliseconds after which the s3 listing is likely to be consistent.
S3CsvSource.Builder	maxFilesInBand(Integer maxFilesInBand) This option specifies the maximum number of files to save from the last maxBand seconds.
S3CsvSource.Builder	multiline(Boolean multiline) A Boolean value that specifies whether a single record can span multiple lines.
S3CsvSource.Builder	name(String name) The name of the data store.
S3CsvSource.Builder	optimizePerformance(Boolean optimizePerformance) A Boolean value that specifies whether to use the advanced SIMD CSV reader along with Apache Arrow based columnar memory formats.
S3CsvSource.Builder	outputSchemas(Collection<GlueSchema> outputSchemas) Specifies the data schema for the S3 CSV source.
S3CsvSource.Builder	outputSchemas(Consumer<GlueSchema.Builder>... outputSchemas) Specifies the data schema for the S3 CSV source.
S3CsvSource.Builder	outputSchemas(GlueSchema... outputSchemas) Specifies the data schema for the S3 CSV source.
S3CsvSource.Builder	paths(Collection<String> paths) A list of the Amazon S3 paths to read from.
S3CsvSource.Builder	paths(String... paths) A list of the Amazon S3 paths to read from.
S3CsvSource.Builder	quoteChar(QuoteChar quoteChar) Specifies the character to use for quoting.
S3CsvSource.Builder	quoteChar(String quoteChar) Specifies the character to use for quoting.
S3CsvSource.Builder	recurse(Boolean recurse) If set to true, recursively reads files in all subdirectories under the specified paths.
S3CsvSource.Builder	separator(Separator separator) Specifies the delimiter character.
S3CsvSource.Builder	separator(String separator) Specifies the delimiter character.
S3CsvSource.Builder	skipFirst(Boolean skipFirst) A Boolean value that specifies whether to skip the first data line.
S3CsvSource.Builder	withHeader(Boolean withHeader) A Boolean value that specifies whether to treat the first line as a header.
S3CsvSource.Builder	writeHeader(Boolean writeHeader) A Boolean value that specifies whether to write the header to output.

Methods inherited from interface software.amazon.awssdk.core.SdkPojo

equalsBySdkFields, sdkFields

Methods inherited from interface software.amazon.awssdk.utils.builder.CopyableBuilder

copy

Methods inherited from interface software.amazon.awssdk.utils.builder.SdkBuilder

applyMutation, build

Method Detail

name

S3CsvSource.Builder name(String name)

The name of the data store.

Parameters:

name - The name of the data store.

Returns:

Returns a reference to this object so that method calls can be chained together.

paths

S3CsvSource.Builder paths(Collection<String> paths)

A list of the Amazon S3 paths to read from.

Parameters:

paths - A list of the Amazon S3 paths to read from.

Returns:

Returns a reference to this object so that method calls can be chained together.

paths

S3CsvSource.Builder paths(String... paths)

A list of the Amazon S3 paths to read from.

Parameters:

paths - A list of the Amazon S3 paths to read from.

Returns:

Returns a reference to this object so that method calls can be chained together.

compressionType

S3CsvSource.Builder compressionType(String compressionType)

Specifies how the data is compressed. This is generally not necessary if the data has a standard file extension. Possible values are "gzip" and "bzip").

Parameters:

compressionType - Specifies how the data is compressed. This is generally not necessary if the data has a standard file extension. Possible values are "gzip" and "bzip").

Returns:

Returns a reference to this object so that method calls can be chained together.

See Also:

CompressionType, CompressionType

compressionType

S3CsvSource.Builder compressionType(CompressionType compressionType)

Specifies how the data is compressed. This is generally not necessary if the data has a standard file extension. Possible values are "gzip" and "bzip").

Parameters:

compressionType - Specifies how the data is compressed. This is generally not necessary if the data has a standard file extension. Possible values are "gzip" and "bzip").

Returns:

Returns a reference to this object so that method calls can be chained together.

See Also:

CompressionType, CompressionType

exclusions

S3CsvSource.Builder exclusions(Collection<String> exclusions)

A string containing a JSON list of Unix-style glob patterns to exclude. For example, "[\"**.pdf\"]" excludes all PDF files.

Parameters:

exclusions - A string containing a JSON list of Unix-style glob patterns to exclude. For example, "[\"**.pdf\"]" excludes all PDF files.

Returns:

Returns a reference to this object so that method calls can be chained together.

exclusions

S3CsvSource.Builder exclusions(String... exclusions)

A string containing a JSON list of Unix-style glob patterns to exclude. For example, "[\"**.pdf\"]" excludes all PDF files.

Parameters:

exclusions - A string containing a JSON list of Unix-style glob patterns to exclude. For example, "[\"**.pdf\"]" excludes all PDF files.

Returns:

Returns a reference to this object so that method calls can be chained together.

groupSize

S3CsvSource.Builder groupSize(String groupSize)

The target group size in bytes. The default is computed based on the input data size and the size of your cluster. When there are fewer than 50,000 input files, "groupFiles" must be set to "inPartition" for this to take effect.

Parameters:

groupSize - The target group size in bytes. The default is computed based on the input data size and the size of your cluster. When there are fewer than 50,000 input files, "groupFiles" must be set to "inPartition" for this to take effect.

Returns:

Returns a reference to this object so that method calls can be chained together.

groupFiles

S3CsvSource.Builder groupFiles(String groupFiles)

Grouping files is turned on by default when the input contains more than 50,000 files. To turn on grouping with fewer than 50,000 files, set this parameter to "inPartition". To disable grouping when there are more than 50,000 files, set this parameter to "none".

Parameters:

groupFiles - Grouping files is turned on by default when the input contains more than 50,000 files. To turn on grouping with fewer than 50,000 files, set this parameter to "inPartition". To disable grouping when there are more than 50,000 files, set this parameter to "none".

Returns:

Returns a reference to this object so that method calls can be chained together.

recurse

S3CsvSource.Builder recurse(Boolean recurse)

If set to true, recursively reads files in all subdirectories under the specified paths.

Parameters:

recurse - If set to true, recursively reads files in all subdirectories under the specified paths.

Returns:

Returns a reference to this object so that method calls can be chained together.

maxBand

S3CsvSource.Builder maxBand(Integer maxBand)

This option controls the duration in milliseconds after which the s3 listing is likely to be consistent. Files with modification timestamps falling within the last maxBand milliseconds are tracked specially when using JobBookmarks to account for Amazon S3 eventual consistency. Most users don't need to set this option. The default is 900000 milliseconds, or 15 minutes.

Parameters:

maxBand - This option controls the duration in milliseconds after which the s3 listing is likely to be consistent. Files with modification timestamps falling within the last maxBand milliseconds are tracked specially when using JobBookmarks to account for Amazon S3 eventual consistency. Most users don't need to set this option. The default is 900000 milliseconds, or 15 minutes.

Returns:

Returns a reference to this object so that method calls can be chained together.

maxFilesInBand

S3CsvSource.Builder maxFilesInBand(Integer maxFilesInBand)

This option specifies the maximum number of files to save from the last maxBand seconds. If this number is exceeded, extra files are skipped and only processed in the next job run.

Parameters:

maxFilesInBand - This option specifies the maximum number of files to save from the last maxBand seconds. If this number is exceeded, extra files are skipped and only processed in the next job run.

Returns:

Returns a reference to this object so that method calls can be chained together.

additionalOptions

S3CsvSource.Builder additionalOptions(S3DirectSourceAdditionalOptions additionalOptions)

Specifies additional connection options.

Parameters:

additionalOptions - Specifies additional connection options.

Returns:

Returns a reference to this object so that method calls can be chained together.

additionalOptions

default S3CsvSource.Builder additionalOptions(Consumer<S3DirectSourceAdditionalOptions.Builder> additionalOptions)

Specifies additional connection options.

This is a convenience method that creates an instance of the S3DirectSourceAdditionalOptions.Builder avoiding the need to create one manually via S3DirectSourceAdditionalOptions.builder().

When the Consumer completes, SdkBuilder.build() is called immediately and its result is passed to additionalOptions(S3DirectSourceAdditionalOptions).

Parameters:

additionalOptions - a consumer that will call methods on S3DirectSourceAdditionalOptions.Builder

Returns:

Returns a reference to this object so that method calls can be chained together.

See Also:

additionalOptions(S3DirectSourceAdditionalOptions)

separator

S3CsvSource.Builder separator(String separator)

Specifies the delimiter character. The default is a comma: ",", but any other character can be specified.

Parameters:

separator - Specifies the delimiter character. The default is a comma: ",", but any other character can be specified.

Returns:

Returns a reference to this object so that method calls can be chained together.

See Also:

Separator, Separator

separator

S3CsvSource.Builder separator(Separator separator)

Specifies the delimiter character. The default is a comma: ",", but any other character can be specified.

Parameters:

separator - Specifies the delimiter character. The default is a comma: ",", but any other character can be specified.

Returns:

Returns a reference to this object so that method calls can be chained together.

See Also:

Separator, Separator

escaper

S3CsvSource.Builder escaper(String escaper)

Specifies a character to use for escaping. This option is used only when reading CSV files. The default value is none. If enabled, the character which immediately follows is used as-is, except for a small set of well-known escapes (\n, \r, \t, and \0).

Parameters:

escaper - Specifies a character to use for escaping. This option is used only when reading CSV files. The default value is none. If enabled, the character which immediately follows is used as-is, except for a small set of well-known escapes (\n, \r, \t, and \0).

Returns:

Returns a reference to this object so that method calls can be chained together.

quoteChar

S3CsvSource.Builder quoteChar(String quoteChar)

Specifies the character to use for quoting. The default is a double quote: '"'. Set this to -1 to turn off quoting entirely.

Parameters:

quoteChar - Specifies the character to use for quoting. The default is a double quote: '"'. Set this to -1 to turn off quoting entirely.

Returns:

Returns a reference to this object so that method calls can be chained together.

See Also:

QuoteChar, QuoteChar

quoteChar

S3CsvSource.Builder quoteChar(QuoteChar quoteChar)

Specifies the character to use for quoting. The default is a double quote: '"'. Set this to -1 to turn off quoting entirely.

Parameters:

quoteChar - Specifies the character to use for quoting. The default is a double quote: '"'. Set this to -1 to turn off quoting entirely.

Returns:

Returns a reference to this object so that method calls can be chained together.

See Also:

QuoteChar, QuoteChar

multiline

S3CsvSource.Builder multiline(Boolean multiline)

A Boolean value that specifies whether a single record can span multiple lines. This can occur when a field contains a quoted new-line character. You must set this option to True if any record spans multiple lines. The default value is False, which allows for more aggressive file-splitting during parsing.

Parameters:

multiline - A Boolean value that specifies whether a single record can span multiple lines. This can occur when a field contains a quoted new-line character. You must set this option to True if any record spans multiple lines. The default value is False, which allows for more aggressive file-splitting during parsing.

Returns:

Returns a reference to this object so that method calls can be chained together.

withHeader

S3CsvSource.Builder withHeader(Boolean withHeader)

A Boolean value that specifies whether to treat the first line as a header. The default value is False.

Parameters:

withHeader - A Boolean value that specifies whether to treat the first line as a header. The default value is False.

Returns:

Returns a reference to this object so that method calls can be chained together.

writeHeader

S3CsvSource.Builder writeHeader(Boolean writeHeader)

A Boolean value that specifies whether to write the header to output. The default value is True.

Parameters:

writeHeader - A Boolean value that specifies whether to write the header to output. The default value is True.

Returns:

Returns a reference to this object so that method calls can be chained together.

skipFirst

S3CsvSource.Builder skipFirst(Boolean skipFirst)

A Boolean value that specifies whether to skip the first data line. The default value is False.

Parameters:

skipFirst - A Boolean value that specifies whether to skip the first data line. The default value is False.

Returns:

Returns a reference to this object so that method calls can be chained together.

optimizePerformance

S3CsvSource.Builder optimizePerformance(Boolean optimizePerformance)

A Boolean value that specifies whether to use the advanced SIMD CSV reader along with Apache Arrow based columnar memory formats. Only available in Glue version 3.0.

Parameters:

optimizePerformance - A Boolean value that specifies whether to use the advanced SIMD CSV reader along with Apache Arrow based columnar memory formats. Only available in Glue version 3.0.

Returns:

Returns a reference to this object so that method calls can be chained together.

outputSchemas

S3CsvSource.Builder outputSchemas(Collection<GlueSchema> outputSchemas)

Specifies the data schema for the S3 CSV source.

Parameters:

outputSchemas - Specifies the data schema for the S3 CSV source.

Returns:

Returns a reference to this object so that method calls can be chained together.

outputSchemas

S3CsvSource.Builder outputSchemas(GlueSchema... outputSchemas)

Specifies the data schema for the S3 CSV source.

Parameters:

outputSchemas - Specifies the data schema for the S3 CSV source.

Returns:

Returns a reference to this object so that method calls can be chained together.

outputSchemas

S3CsvSource.Builder outputSchemas(Consumer<GlueSchema.Builder>... outputSchemas)

Specifies the data schema for the S3 CSV source.

This is a convenience method that creates an instance of the GlueSchema.Builder avoiding the need to create one manually via GlueSchema.builder().

When the Consumer completes, SdkBuilder.build() is called immediately and its result is passed to #outputSchemas(List).

Parameters:

outputSchemas - a consumer that will call methods on GlueSchema.Builder

Returns:

Returns a reference to this object so that method calls can be chained together.

See Also:

#outputSchemas(java.util.Collection)