Skip to content

< Back

LLamaWeights

Namespace: LLama

A set of model weights, loaded into memory.

1
public sealed class LLamaWeights : System.IDisposable

Inheritance Object → LLamaWeights
Implements IDisposable
Attributes NullableContextAttribute, NullableAttribute

Properties

NativeHandle

The native handle, which is used in the native APIs

1
public SafeLlamaModelHandle NativeHandle { get; }

Property Value

SafeLlamaModelHandle

Remarks:

Be careful how you use this!

ContextSize

Total number of tokens in the context

1
public int ContextSize { get; }

Property Value

Int32

SizeInBytes

Get the size of this model in bytes

1
public ulong SizeInBytes { get; }

Property Value

UInt64

ParameterCount

Get the number of parameters in this model

1
public ulong ParameterCount { get; }

Property Value

UInt64

EmbeddingSize

Dimension of embedding vectors

1
public int EmbeddingSize { get; }

Property Value

Int32

Vocab

Get the special tokens of this model

1
public Vocabulary Vocab { get; }

Property Value

Vocabulary

Metadata

All metadata keys in this model

1
public IReadOnlyDictionary<string, string> Metadata { get; set; }

Property Value

IReadOnlyDictionary<String, String>

Methods

LoadFromFile(IModelParams)

Load weights into memory

1
public static LLamaWeights LoadFromFile(IModelParams params)

Parameters

params IModelParams

Returns

LLamaWeights

LoadFromFileAsync(IModelParams, CancellationToken, IProgress<Single>)

Load weights into memory

1
public static Task<LLamaWeights> LoadFromFileAsync(IModelParams params, CancellationToken token, IProgress<float> progressReporter)

Parameters

params IModelParams
Parameters to use to load the model

token CancellationToken
A cancellation token that can interrupt model loading

progressReporter IProgress<Single>
Receives progress updates as the model loads (0 to 1)

Returns

Task<LLamaWeights>

Exceptions

LoadWeightsFailedException
Thrown if weights failed to load for any reason. e.g. Invalid file format or loading cancelled.

OperationCanceledException
Thrown if the cancellation token is cancelled.

Dispose()

1
public void Dispose()

CreateContext(IContextParams, ILogger)

Create a llama_context using this model

1
public LLamaContext CreateContext(IContextParams params, ILogger logger)

Parameters

params IContextParams

logger ILogger

Returns

LLamaContext

Tokenize(String, Boolean, Boolean, Encoding)

Convert a string of text into tokens

1
public LLamaToken[] Tokenize(string text, bool add_bos, bool special, Encoding encoding)

Parameters

text String

add_bos Boolean

special Boolean
Allow tokenizing special and/or control tokens which otherwise are not exposed and treated as plaintext.

encoding Encoding

Returns

LLamaToken[]

< Back