com.plexpt.chatgpt.util

Class TokensUtil

java.lang.Object

com.plexpt.chatgpt.util.TokensUtil

public class TokensUtil
extends Object

Field Summary

Fields

Modifier and Type	Field and Description
private static com.knuddels.jtokkit.api.EncodingRegistry	encodingRegistry
private static Map<String,com.knuddels.jtokkit.api.Encoding>	modelEncodingMap

Constructor Summary

Constructors

Constructor and Description
TokensUtil()

Method Summary

Modifier and Type	Method and Description
static int	tokens(String modelName, List<Message> messages) 计算tokens

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

modelEncodingMap

private static final Map<String,com.knuddels.jtokkit.api.Encoding> modelEncodingMap

encodingRegistry

private static final com.knuddels.jtokkit.api.EncodingRegistry encodingRegistry

Constructor Detail

TokensUtil

public TokensUtil()

Method Detail

tokens

public static int tokens(String modelName,
                         List<Message> messages)

计算tokens

Parameters:

modelName - 模型名称

messages - 消息列表

Returns:

计算出的tokens数量