StreamingTokenDecoder
Namespace: LLama
Decodes a stream of tokens into a stream of characters
public sealed class StreamingTokenDecoder
Inheritance Object → StreamingTokenDecoder
Properties
AvailableCharacters
The number of decoded characters waiting to be read
public int AvailableCharacters { get; }
Property Value
Constructors
StreamingTokenDecoder(Encoding, LLamaWeights)
Create a new decoder
public StreamingTokenDecoder(Encoding encoding, LLamaWeights weights)
Parameters
encoding Encoding
Text encoding to use
weights LLamaWeights
Model weights
StreamingTokenDecoder(LLamaContext)
Create a new decoder
public StreamingTokenDecoder(LLamaContext context)
Parameters
context LLamaContext
Context to retrieve encoding and model weights from
StreamingTokenDecoder(Encoding, SafeLLamaContextHandle)
Create a new decoder
public StreamingTokenDecoder(Encoding encoding, SafeLLamaContextHandle context)
Parameters
encoding Encoding
Text encoding to use
context SafeLLamaContextHandle
Context to retrieve model weights from
StreamingTokenDecoder(Encoding, SafeLlamaModelHandle)
Create a new decoder
public StreamingTokenDecoder(Encoding encoding, SafeLlamaModelHandle weights)
Parameters
encoding Encoding
Text encoding to use
weights SafeLlamaModelHandle
Models weights to use
Methods
Add(LLamaToken)
Add a single token to the decoder
public void Add(LLamaToken token)
Parameters
token LLamaToken
Add(Int32)
Add a single token to the decoder
public void Add(int token)
Parameters
token Int32
AddRange<T>(T)
Add all tokens in the given enumerable
public void AddRange<T>(T tokens)
Type Parameters
T
Parameters
tokens T
AddRange(ReadOnlySpan<LLamaToken>)
Add all tokens in the given span
public void AddRange(ReadOnlySpan<LLamaToken> tokens)
Parameters
tokens ReadOnlySpan<LLamaToken>
Read(List<Char>)
Read all decoded characters and clear the buffer
public void Read(List<char> dest)
Parameters
dest List<Char>
Read()
Read all decoded characters as a string and clear the buffer
public string Read()
Returns
Reset()
Set the decoder back to its initial state
public void Reset()
<Add>g__TokenToBytes|9_0(Byte[]&, LLamaToken, SafeLlamaModelHandle)
internal static Span<byte> <Add>g__TokenToBytes|9_0(Byte[]& bytes, LLamaToken token, SafeLlamaModelHandle model)
Parameters
bytes Byte[]&
token LLamaToken
model SafeLlamaModelHandle