LLamaModel
Namespace: LLama
The abstraction of a LLama model, which holds the context in the native library.
public class LLamaModel : System.IDisposable
Inheritance Object → LLamaModel
Implements IDisposable
Properties
ContextSize
The context size.
public int ContextSize { get; }
Property Value
Params
The model params set for this model.
public ModelParams Params { get; set; }
Property Value
NativeHandle
The native handle, which is used to be passed to the native APIs. Please avoid using it unless you know what is the usage of the Native API.
public SafeLLamaContextHandle NativeHandle { get; }
Property Value
Encoding
The encoding set for this model to deal with text input.
public Encoding Encoding { get; }
Property Value
Constructors
LLamaModel(ModelParams, String, ILLamaLogger)
public LLamaModel(ModelParams Params, string encoding, ILLamaLogger logger)
Parameters
Params ModelParams
Model params.
encoding String
Encoding to deal with text input.
logger ILLamaLogger
The logger.
Methods
Tokenize(String, Boolean)
Tokenize a string.
public IEnumerable<int> Tokenize(string text, bool addBos)
Parameters
text String
addBos Boolean
Whether to add a bos to the text.
Returns
DeTokenize(IEnumerable<Int32>)
Detokenize the tokens to text.
public string DeTokenize(IEnumerable<int> tokens)
Parameters
tokens IEnumerable<Int32>
Returns
SaveState(String)
Save the state to specified path.
public void SaveState(string filename)
Parameters
filename String
GetStateData()
Get the state data as a byte array.
public Byte[] GetStateData()
Returns
LoadState(String)
Load the state from specified path.
public void LoadState(string filename)
Parameters
filename String
Exceptions
LoadState(Byte[])
Load the state from memory.
public void LoadState(Byte[] stateData)
Parameters
stateData Byte[]
Exceptions
Sample(LLamaTokenDataArray, Single, MiroStateType, Single, Single, Int32, Single, Single, Single)
Perform the sampling. Please don't use it unless you fully know what it does.
public int Sample(LLamaTokenDataArray candidates, float temperature, MiroStateType mirostat, float mirostatTau, float mirostatEta, int topK, float topP, float tfsZ, float typicalP)
Parameters
candidates LLamaTokenDataArray
temperature Single
mirostat MiroStateType
mirostatTau Single
mirostatEta Single
topK Int32
topP Single
tfsZ Single
typicalP Single
Returns
ApplyPenalty(IEnumerable<Int32>, Dictionary<Int32, Single>, Int32, Single, Single, Single, Boolean)
Apply the penalty for the tokens. Please don't use it unless you fully know what it does.
public LLamaTokenDataArray ApplyPenalty(IEnumerable<int> lastTokens, Dictionary<int, float> logitBias, int repeatLastTokensCount, float repeatPenalty, float alphaFrequency, float alphaPresence, bool penalizeNL)
Parameters
lastTokens IEnumerable<Int32>
logitBias Dictionary<Int32, Single>
repeatLastTokensCount Int32
repeatPenalty Single
alphaFrequency Single
alphaPresence Single
penalizeNL Boolean
Returns
Eval(Int32[], Int32)
public int Eval(Int32[] tokens, int pastTokensCount)
Parameters
tokens Int32[]
pastTokensCount Int32
Returns
Int32
The updated pastTokensCount.
Exceptions
GenerateResult(IEnumerable<Int32>)
internal IEnumerable<string> GenerateResult(IEnumerable<int> ids)
Parameters
Returns
Dispose()
public void Dispose()