org.apache.arrow.dataset.file

Class DatasetFileWriter

java.lang.Object

org.apache.arrow.dataset.file.DatasetFileWriter

public class DatasetFileWriter
extends Object

JNI-based utility to write datasets into files. It internally depends on C++ static method FileSystemDataset::Write.

Constructor Summary

Constructors

Constructor and Description
DatasetFileWriter()

Method Summary

Modifier and Type	Method and Description
static void	write(BufferAllocator allocator, ArrowReader reader, FileFormat format, String uri) Write the contents of an ArrowReader as a dataset, with default partitioning settings.
static void	write(BufferAllocator allocator, ArrowReader reader, FileFormat format, String uri, String[] partitionColumns, int maxPartitions, String baseNameTemplate) Write the contents of an ArrowReader as a dataset.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DatasetFileWriter

public DatasetFileWriter()

Method Detail

write

public static void write(BufferAllocator allocator,
                         ArrowReader reader,
                         FileFormat format,
                         String uri,
                         String[] partitionColumns,
                         int maxPartitions,
                         String baseNameTemplate)

Write the contents of an ArrowReader as a dataset.

Parameters:

reader - the datasource for writing

format - target file format

uri - target file uri

maxPartitions - maximum partitions to be included in written files

partitionColumns - columns used to partition output files. Empty to disable partitioning

baseNameTemplate - file name template used to make partitions. E.g. "dat_{i}", i is current partition ID around all written files.

write

public static void write(BufferAllocator allocator,
                         ArrowReader reader,
                         FileFormat format,
                         String uri)

Write the contents of an ArrowReader as a dataset, with default partitioning settings.

Parameters:

reader - the datasource for writing

format - target file format

uri - target file uri