SafeLlamaModelHandle
Namespace: LLama.Native
A reference to a set of llama model weights
public sealed class SafeLlamaModelHandle : SafeLLamaHandleBase, System.IDisposable
Inheritance Object → CriticalFinalizerObject → SafeHandle → SafeLLamaHandleBase → SafeLlamaModelHandle
Implements IDisposable
Properties
VocabCount
Total number of tokens in vocabulary of this model
public int VocabCount { get; }
Property Value
ContextSize
Total number of tokens in the context
public int ContextSize { get; }
Property Value
EmbeddingSize
Dimension of embedding vectors
public int EmbeddingSize { get; }
Property Value
IsInvalid
public bool IsInvalid { get; }
Property Value
IsClosed
public bool IsClosed { get; }
Property Value
Methods
ReleaseHandle()
protected bool ReleaseHandle()
Returns
LoadFromFile(String, LLamaContextParams)
Load a model from the given file path into memory
public static SafeLlamaModelHandle LoadFromFile(string modelPath, LLamaContextParams lparams)
Parameters
modelPath String
lparams LLamaContextParams
Returns
Exceptions
ApplyLoraFromFile(String, String, Int32)
Apply a LoRA adapter to a loaded model
public void ApplyLoraFromFile(string lora, string modelBase, int threads)
Parameters
lora String
modelBase String
A path to a higher quality model to use as a base for the layers modified by the
adapter. Can be NULL to use the current loaded model.
threads Int32
Exceptions
TokenToSpan(Int32, Span<Byte>)
Convert a single llama token into bytes
public int TokenToSpan(int llama_token, Span<byte> dest)
Parameters
llama_token Int32
Token to decode
dest Span<Byte>
A span to attempt to write into. If this is too small nothing will be written
Returns
Int32
The size of this token. nothing will be written if this is larger than dest
TokenToString(Int32, Encoding)
Convert a single llama token into a string
public string TokenToString(int llama_token, Encoding encoding)
Parameters
llama_token Int32
encoding Encoding
Encoding to use to decode the bytes into a string
Returns
TokenToString(Int32, Encoding, StringBuilder)
Append a single llama token to a string builder
public void TokenToString(int llama_token, Encoding encoding, StringBuilder dest)
Parameters
llama_token Int32
Token to decode
encoding Encoding
dest StringBuilder
string builder to append the result to
Tokenize(String, Boolean, Encoding)
Convert a string of text into tokens
public Int32[] Tokenize(string text, bool add_bos, Encoding encoding)
Parameters
text String
add_bos Boolean
encoding Encoding
Returns
CreateContext(LLamaContextParams)
Create a new context for this model
public SafeLLamaContextHandle CreateContext(LLamaContextParams params)
Parameters
params LLamaContextParams