org.apache.flink.connector.mongodb.source

Class MongoSourceBuilder<OUT>

java.lang.Object

org.apache.flink.connector.mongodb.source.MongoSourceBuilder<OUT>

Type Parameters:

OUT - The output type of the source.

@PublicEvolving
public class MongoSourceBuilder<OUT>
extends Object

The builder class for MongoSource to make it easier for the users to construct a MongoSource.

Method Summary

Modifier and Type	Method and Description
MongoSource<OUT>	build() Build the MongoSource.
MongoSourceBuilder<OUT>	setCollection(String collection) Sets the collection to sink of MongoDB.
MongoSourceBuilder<OUT>	setDatabase(String database) Sets the database to sink of MongoDB.
MongoSourceBuilder<OUT>	setDeserializationSchema(MongoDeserializationSchema<OUT> deserializationSchema) Sets the deserialization schema for MongoDB BsonDocument.
MongoSourceBuilder<OUT>	setFetchSize(int fetchSize) Sets the number of documents should be fetched per round-trip when reading.
MongoSourceBuilder<OUT>	setFilter(org.bson.conversions.Bson filter) Sets the filter of documents to read.
MongoSourceBuilder<OUT>	setLimit(int limit) Sets the limit of documents to read.
MongoSourceBuilder<OUT>	setNoCursorTimeout(boolean noCursorTimeout) The MongoDB server normally times out idle cursors after an inactivity period (10 minutes) to prevent excess memory use.
MongoSourceBuilder<OUT>	setPartitionSize(org.apache.flink.configuration.MemorySize partitionSize) Sets the partition memory size of MongoDB split.
MongoSourceBuilder<OUT>	setPartitionStrategy(PartitionStrategy partitionStrategy) Sets the partition strategy.
MongoSourceBuilder<OUT>	setProjectedFields(List<String> projectedFields) Sets the projection fields of documents to read.
MongoSourceBuilder<OUT>	setProjectedFields(String... projectedFields) Sets the projection fields of documents to read.
MongoSourceBuilder<OUT>	setSamplesPerPartition(int samplesPerPartition) Sets the number of samples to take per partition which is only used for the sample partition strategy PartitionStrategy.SAMPLE.
MongoSourceBuilder<OUT>	setUri(String uri) Sets the connection string of MongoDB.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

setUri

public MongoSourceBuilder<OUT> setUri(String uri)

Sets the connection string of MongoDB.

Parameters:

uri - connection string of MongoDB

Returns:

this builder

setDatabase

public MongoSourceBuilder<OUT> setDatabase(String database)

Sets the database to sink of MongoDB.

Parameters:

database - the database to read from MongoDB.

Returns:

this builder

setCollection

public MongoSourceBuilder<OUT> setCollection(String collection)

Sets the collection to sink of MongoDB.

Parameters:

collection - the collection to read from MongoDB.

Returns:

this builder

setFetchSize

public MongoSourceBuilder<OUT> setFetchSize(int fetchSize)

Sets the number of documents should be fetched per round-trip when reading.

Parameters:

fetchSize - the number of documents should be fetched per round-trip when reading.

Returns:

this builder

setNoCursorTimeout

public MongoSourceBuilder<OUT> setNoCursorTimeout(boolean noCursorTimeout)

The MongoDB server normally times out idle cursors after an inactivity period (10 minutes) to prevent excess memory use. Set this option to prevent that. If a session is idle for longer than 30 minutes, the MongoDB server marks that session as expired and may close it at any time. When the MongoDB server closes the session, it also kills any in-progress operations and open cursors associated with the session. This includes cursors configured with noCursorTimeout() or a maxTimeMS() greater than 30 minutes.

Parameters:

noCursorTimeout - Set this option to true to prevent cursor timeout (10 minutes)

Returns:

this builder

setPartitionStrategy

public MongoSourceBuilder<OUT> setPartitionStrategy(PartitionStrategy partitionStrategy)

Sets the partition strategy. Available partition strategies are single, sample, split-vector, sharded and default. You can see PartitionStrategy for detail.

Parameters:

partitionStrategy - the strategy of a partition.

Returns:

this builder

setPartitionSize

public MongoSourceBuilder<OUT> setPartitionSize(org.apache.flink.configuration.MemorySize partitionSize)

Sets the partition memory size of MongoDB split. Split a MongoDB collection into multiple partitions according to the partition memory size. Partitions can be read in parallel by multiple MongoScanSourceSplitReader to speed up the overall read time.

Parameters:

partitionSize - the memory size of a partition.

Returns:

this builder

setSamplesPerPartition

public MongoSourceBuilder<OUT> setSamplesPerPartition(int samplesPerPartition)

Sets the number of samples to take per partition which is only used for the sample partition strategy PartitionStrategy.SAMPLE. The sample partitioner samples the collection, projects and sorts by the partition fields. Then uses every samplesPerPartition as the value to use to calculate the partition boundaries. The total number of samples taken is: samples per partition * ( count of documents / number of documents per partition).

Parameters:

samplesPerPartition - number of samples per partition.

Returns:

this builder

setLimit

public MongoSourceBuilder<OUT> setLimit(int limit)

Sets the limit of documents to read. If limit is not set or set to -1, the documents of the entire collection will be read.

Parameters:

limit - the limit of documents to read.

Returns:

this builder

setFilter

public MongoSourceBuilder<OUT> setFilter(org.bson.conversions.Bson filter)

Sets the filter of documents to read.

Parameters:

filter - the filter of documents to read.

Returns:

this builder

setProjectedFields

public MongoSourceBuilder<OUT> setProjectedFields(String... projectedFields)

Sets the projection fields of documents to read.

Parameters:

projectedFields - the projection fields of documents to read.

Returns:

this builder

setProjectedFields

public MongoSourceBuilder<OUT> setProjectedFields(List<String> projectedFields)

Sets the projection fields of documents to read.

Parameters:

projectedFields - the projection fields of documents to read.

Returns:

this builder

setDeserializationSchema

public MongoSourceBuilder<OUT> setDeserializationSchema(MongoDeserializationSchema<OUT> deserializationSchema)

Sets the deserialization schema for MongoDB BsonDocument.

Parameters:

deserializationSchema - the deserialization schema to deserialize BsonDocument.

Returns:

this builder

build

public MongoSource<OUT> build()

Build the MongoSource.

Returns:

a MongoSource with the settings made for this builder.