Package ai.vespa.llm

Class InferenceParameters

java.lang.Object

ai.vespa.llm.InferenceParameters

public class InferenceParameters
extends Object

Parameters for inference to language models. Parameters are typically supplied from searchers or processors and comes from query strings, headers, or other sources. Which parameters are available depends on the language model used. author lesters author glebashnik

Field Summary

Fields

Modifier and Type	Field	Description
static final String	OPTION_FREQUENCY_PENALTY
static final String	OPTION_JSON_SCHEMA
static final String	OPTION_MAX_TOKENS
static final String	OPTION_MODEL
static final String	OPTION_N_PREDICT
static final String	OPTION_PRESENCE_PENALTY
static final String	OPTION_REPEAT_PENALTY
static final String	OPTION_SEED
static final String	OPTION_TEMPERATURE
static final String	OPTION_TOP_K
static final String	OPTION_TOP_P

Constructor Summary

Constructors

Constructor	Description
InferenceParameters(String apiKey, String endpoint, Function<String,String> options)
InferenceParameters(String apiKey, Function<String,String> options)
InferenceParameters(Function<String,String> options)

Method Summary

Modifier and Type	Method	Description
Optional<String>	get(String option)
Optional<String>	getApiKey()
Optional<Double>	getDouble(String option)
Optional<String>	getEndpoint()
Optional<Integer>	getInt(String option)
Optional<Long>	getLong(String option)
void	ifPresent(String option, Consumer<String> func)
void	setApiKey(String apiKey)
void	setEndpoint(String endpoint)
InferenceParameters	withDefaultOptions(Function<String,String> defaultOptions)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

OPTION_MODEL

public static final String OPTION_MODEL

See Also:

• Constant Field Values

OPTION_TEMPERATURE

public static final String OPTION_TEMPERATURE

See Also:

• Constant Field Values

OPTION_MAX_TOKENS

public static final String OPTION_MAX_TOKENS

See Also:

• Constant Field Values

OPTION_TOP_K

public static final String OPTION_TOP_K

See Also:

• Constant Field Values

OPTION_TOP_P

public static final String OPTION_TOP_P

See Also:

• Constant Field Values

OPTION_N_PREDICT

public static final String OPTION_N_PREDICT

See Also:

• Constant Field Values

OPTION_REPEAT_PENALTY

public static final String OPTION_REPEAT_PENALTY

See Also:

• Constant Field Values

OPTION_FREQUENCY_PENALTY

public static final String OPTION_FREQUENCY_PENALTY

See Also:

• Constant Field Values

OPTION_PRESENCE_PENALTY

public static final String OPTION_PRESENCE_PENALTY

See Also:

• Constant Field Values

OPTION_SEED

public static final String OPTION_SEED

See Also:

• Constant Field Values

OPTION_JSON_SCHEMA

public static final String OPTION_JSON_SCHEMA

See Also:

• Constant Field Values

Constructor Details

InferenceParameters

public InferenceParameters(Function<String,String> options)

InferenceParameters

public InferenceParameters(String apiKey,
 Function<String,String> options)

InferenceParameters

public InferenceParameters(String apiKey,
 String endpoint,
 Function<String,String> options)

Method Details

setApiKey

public void setApiKey(String apiKey)

getApiKey

public Optional<String> getApiKey()

setEndpoint

public void setEndpoint(String endpoint)

getEndpoint

public Optional<String> getEndpoint()

get

public Optional<String> get(String option)

getDouble

public Optional<Double> getDouble(String option)

getInt

public Optional<Integer> getInt(String option)

getLong

public Optional<Long> getLong(String option)

ifPresent

public void ifPresent(String option,
 Consumer<String> func)

withDefaultOptions

public InferenceParameters withDefaultOptions(Function<String,String> defaultOptions)