Skip to content

StreamingTokenDecoder

Namespace: LLama

Decodes a stream of tokens into a stream of characters

1
public sealed class StreamingTokenDecoder

Inheritance Object → StreamingTokenDecoder

Properties

AvailableCharacters

The number of decoded characters waiting to be read

1
public int AvailableCharacters { get; }

Property Value

Int32

Constructors

StreamingTokenDecoder(Encoding, LLamaWeights)

Create a new decoder

1
public StreamingTokenDecoder(Encoding encoding, LLamaWeights weights)

Parameters

encoding Encoding
Text encoding to use

weights LLamaWeights
Model weights

StreamingTokenDecoder(LLamaContext)

Create a new decoder

1
public StreamingTokenDecoder(LLamaContext context)

Parameters

context LLamaContext
Context to retrieve encoding and model weights from

StreamingTokenDecoder(Encoding, SafeLLamaContextHandle)

Create a new decoder

1
public StreamingTokenDecoder(Encoding encoding, SafeLLamaContextHandle context)

Parameters

encoding Encoding
Text encoding to use

context SafeLLamaContextHandle
Context to retrieve model weights from

StreamingTokenDecoder(Encoding, SafeLlamaModelHandle)

Create a new decoder

1
public StreamingTokenDecoder(Encoding encoding, SafeLlamaModelHandle weights)

Parameters

encoding Encoding
Text encoding to use

weights SafeLlamaModelHandle
Models weights to use

Methods

Add(LLamaToken)

Add a single token to the decoder

1
public void Add(LLamaToken token)

Parameters

token LLamaToken

Add(Int32)

Add a single token to the decoder

1
public void Add(int token)

Parameters

token Int32

AddRange<T>(T)

Add all tokens in the given enumerable

1
public void AddRange<T>(T tokens)

Type Parameters

T

Parameters

tokens T

AddRange(ReadOnlySpan<LLamaToken>)

Add all tokens in the given span

1
public void AddRange(ReadOnlySpan<LLamaToken> tokens)

Parameters

tokens ReadOnlySpan<LLamaToken>

Read(List<Char>)

Read all decoded characters and clear the buffer

1
public void Read(List<char> dest)

Parameters

dest List<Char>

Read()

Read all decoded characters as a string and clear the buffer

1
public string Read()

Returns

String

Reset()

Set the decoder back to its initial state

1
public void Reset()

<Add>g__TokenToBytes|9_0(Byte[]&, LLamaToken, SafeLlamaModelHandle)

1
internal static Span<byte> <Add>g__TokenToBytes|9_0(Byte[]& bytes, LLamaToken token, SafeLlamaModelHandle model)

Parameters

bytes Byte[]&

token LLamaToken

model SafeLlamaModelHandle

Returns

Span<Byte>