org.apache.flink.table.sources

Class CsvTableSource

java.lang.Object

org.apache.flink.table.sources.CsvTableSource

All Implemented Interfaces:

BatchTableSource<org.apache.flink.types.Row>, org.apache.flink.table.sources.LookupableTableSource<org.apache.flink.types.Row>, org.apache.flink.table.sources.ProjectableTableSource<org.apache.flink.types.Row>, StreamTableSource<org.apache.flink.types.Row>, org.apache.flink.table.sources.TableSource<org.apache.flink.types.Row>

public class CsvTableSource
extends Object
implements StreamTableSource<org.apache.flink.types.Row>, BatchTableSource<org.apache.flink.types.Row>, org.apache.flink.table.sources.LookupableTableSource<org.apache.flink.types.Row>, org.apache.flink.table.sources.ProjectableTableSource<org.apache.flink.types.Row>

A StreamTableSource and BatchTableSource for simple CSV files with a (logically) unlimited number of fields.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	CsvTableSource.Builder A builder for creating CsvTableSource instances.
static class	CsvTableSource.CsvLookupFunction LookupFunction to support lookup in CsvTableSource.

Constructor Summary

Constructors

Constructor and Description
CsvTableSource(String path, String[] fieldNames, org.apache.flink.api.common.typeinfo.TypeInformation<?>[] fieldTypes) A InputFormatTableSource and LookupableTableSource for simple CSV files with a (logically) unlimited number of fields.
CsvTableSource(String path, String[] fieldNames, org.apache.flink.api.common.typeinfo.TypeInformation<?>[] fieldTypes, int[] selectedFields, String fieldDelim, String lineDelim, Character quoteCharacter, boolean ignoreFirstLine, String ignoreComments, boolean lenient) A InputFormatTableSource and LookupableTableSource for simple CSV files with a (logically) unlimited number of fields.
CsvTableSource(String path, String[] fieldNames, org.apache.flink.api.common.typeinfo.TypeInformation<?>[] fieldTypes, String fieldDelim, String lineDelim, Character quoteCharacter, boolean ignoreFirstLine, String ignoreComments, boolean lenient) A InputFormatTableSource and LookupableTableSource for simple CSV files with a (logically) unlimited number of fields.

Method Summary

Modifier and Type	Method and Description
static CsvTableSource.Builder	builder() Return a new builder that builds a CsvTableSource.
boolean	equals(Object o)
String	explainSource()
org.apache.flink.table.functions.AsyncTableFunction<org.apache.flink.types.Row>	getAsyncLookupFunction(String[] lookupKeys)
org.apache.flink.api.java.DataSet<org.apache.flink.types.Row>	getDataSet(org.apache.flink.api.java.ExecutionEnvironment execEnv) Returns the data of the table as a DataSet.
org.apache.flink.streaming.api.datastream.DataStream<org.apache.flink.types.Row>	getDataStream(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment execEnv) Returns the data of the table as a DataStream.
org.apache.flink.table.functions.TableFunction<org.apache.flink.types.Row>	getLookupFunction(String[] lookupKeys)
org.apache.flink.table.types.DataType	getProducedDataType()
org.apache.flink.table.api.TableSchema	getTableSchema()
int	hashCode()
boolean	isAsyncEnabled()
boolean	isBounded() Returns true if this is a bounded source, false if this is an unbounded source.
CsvTableSource	projectFields(int[] fields)

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.table.sources.TableSource

getReturnType

Constructor Detail

CsvTableSource

public CsvTableSource(String path,
                      String[] fieldNames,
                      org.apache.flink.api.common.typeinfo.TypeInformation<?>[] fieldTypes)

A InputFormatTableSource and LookupableTableSource for simple CSV files with a (logically) unlimited number of fields.

Parameters:

path - The path to the CSV file.

fieldNames - The names of the table fields.

fieldTypes - The types of the table fields.

CsvTableSource

public CsvTableSource(String path,
                      String[] fieldNames,
                      org.apache.flink.api.common.typeinfo.TypeInformation<?>[] fieldTypes,
                      String fieldDelim,
                      String lineDelim,
                      Character quoteCharacter,
                      boolean ignoreFirstLine,
                      String ignoreComments,
                      boolean lenient)

A InputFormatTableSource and LookupableTableSource for simple CSV files with a (logically) unlimited number of fields.

Parameters:

path - The path to the CSV file.

fieldNames - The names of the table fields.

fieldTypes - The types of the table fields.

fieldDelim - The field delimiter, "," by default.

lineDelim - The row delimiter, "\n" by default.

quoteCharacter - An optional quote character for String values, null by default.

ignoreFirstLine - Flag to ignore the first line, false by default.

ignoreComments - An optional prefix to indicate comments, null by default.

lenient - Flag to skip records with parse error instead to fail, false by default.

CsvTableSource

public CsvTableSource(String path,
                      String[] fieldNames,
                      org.apache.flink.api.common.typeinfo.TypeInformation<?>[] fieldTypes,
                      int[] selectedFields,
                      String fieldDelim,
                      String lineDelim,
                      Character quoteCharacter,
                      boolean ignoreFirstLine,
                      String ignoreComments,
                      boolean lenient)

A InputFormatTableSource and LookupableTableSource for simple CSV files with a (logically) unlimited number of fields.

Parameters:

path - The path to the CSV file.

fieldNames - The names of the table fields.

fieldTypes - The types of the table fields.

selectedFields - The fields which will be read and returned by the table source. If None, all fields are returned.

fieldDelim - The field delimiter, "," by default.

lineDelim - The row delimiter, "\n" by default.

quoteCharacter - An optional quote character for String values, null by default.

ignoreFirstLine - Flag to ignore the first line, false by default.

ignoreComments - An optional prefix to indicate comments, null by default.

lenient - Flag to skip records with parse error instead to fail, false by default.

Method Detail

builder

public static CsvTableSource.Builder builder()

Return a new builder that builds a CsvTableSource. For example:

 CsvTableSource source = new CsvTableSource.builder()
     .path("/path/to/your/file.csv")
     .field("myfield", Types.STRING)
     .field("myfield2", Types.INT)
     .build();
 

Returns:

a new builder to build a CsvTableSource

getProducedDataType

public org.apache.flink.table.types.DataType getProducedDataType()

Specified by:

getProducedDataType in interface org.apache.flink.table.sources.TableSource<org.apache.flink.types.Row>

getTableSchema

public org.apache.flink.table.api.TableSchema getTableSchema()

Specified by:

getTableSchema in interface org.apache.flink.table.sources.TableSource<org.apache.flink.types.Row>

projectFields

public CsvTableSource projectFields(int[] fields)

Specified by:

projectFields in interface org.apache.flink.table.sources.ProjectableTableSource<org.apache.flink.types.Row>

isBounded

public boolean isBounded()

Description copied from interface: StreamTableSource

Returns true if this is a bounded source, false if this is an unbounded source. Default is unbounded for compatibility.

Specified by:

isBounded in interface StreamTableSource<org.apache.flink.types.Row>

getDataStream

public org.apache.flink.streaming.api.datastream.DataStream<org.apache.flink.types.Row> getDataStream(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment execEnv)

Description copied from interface: StreamTableSource

Returns the data of the table as a DataStream.

NOTE: This method is for internal use only for defining a TableSource. Do not use it in Table API programs.

Specified by:

getDataStream in interface StreamTableSource<org.apache.flink.types.Row>

getDataSet

public org.apache.flink.api.java.DataSet<org.apache.flink.types.Row> getDataSet(org.apache.flink.api.java.ExecutionEnvironment execEnv)

Description copied from interface: BatchTableSource

Returns the data of the table as a DataSet.

NOTE: This method is for internal use only for defining a TableSource. Do not use it in Table API programs.

Specified by:

getDataSet in interface BatchTableSource<org.apache.flink.types.Row>

getLookupFunction

public org.apache.flink.table.functions.TableFunction<org.apache.flink.types.Row> getLookupFunction(String[] lookupKeys)

Specified by:

getLookupFunction in interface org.apache.flink.table.sources.LookupableTableSource<org.apache.flink.types.Row>

getAsyncLookupFunction

public org.apache.flink.table.functions.AsyncTableFunction<org.apache.flink.types.Row> getAsyncLookupFunction(String[] lookupKeys)

Specified by:

getAsyncLookupFunction in interface org.apache.flink.table.sources.LookupableTableSource<org.apache.flink.types.Row>

isAsyncEnabled

public boolean isAsyncEnabled()

Specified by:

isAsyncEnabled in interface org.apache.flink.table.sources.LookupableTableSource<org.apache.flink.types.Row>

explainSource

public String explainSource()

Specified by:

explainSource in interface org.apache.flink.table.sources.TableSource<org.apache.flink.types.Row>

equals

public boolean equals(Object o)

Overrides:

equals in class Object

hashCode

public int hashCode()

Overrides:

hashCode in class Object