org.apache.flink.connector.mongodb.source.enumerator.splitter

Class MongoSampleSplitter

java.lang.Object

org.apache.flink.connector.mongodb.source.enumerator.splitter.MongoSampleSplitter

@Internal
public class MongoSampleSplitter
extends Object

Sample Partitioner

Samples the collection to generate partitions.

Uses the average document size to split the collection into average sized chunks

The partitioner samples the collection, projects and sorts by the partition fields. Then uses every samplesPerPartition as the value to use to calculate the partition boundaries.

• scan.partition.size: The average size (MB) for each partition. Note: Uses the average document size to determine the number of documents per partition so may not be even. Defaults to: 64mb.
• scan.partition.samples: The number of samples to take per partition. Defaults to: 10. The total number of samples taken is calculated as: samples per partition * (count of documents / number of documents per partition).

Constructor Summary

Constructors

Constructor and Description
MongoSampleSplitter()

Method Summary

Modifier and Type	Method and Description
static Collection<MongoScanSourceSplit>	split(MongoSplitContext splitContext)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MongoSampleSplitter

public MongoSampleSplitter()

Method Detail

split

public static Collection<MongoScanSourceSplit> split(MongoSplitContext splitContext)