ISamplingPipeline

Namespace: LLama.Sampling

Convert a span of logits into a single sampled token. This interface can be implemented to completely customise the sampling process.

public interface ISamplingPipeline : System.IDisposable

Methods

Sample a single token from the given logits

LLamaToken Sample(SafeLLamaContextHandle ctx, Span<float> logits, ReadOnlySpan<LLamaToken> lastTokens)

ctx SafeLLamaContextHandle
The context being sampled from

logits Span<Single>
The logits produced by the model

lastTokens ReadOnlySpan<LLamaToken>
A span of tokens recently returned by the model

Update the pipeline, with knowledge that a particular token was just accepted

void Accept(SafeLLamaContextHandle ctx, LLamaToken token)

Reset all internal state of the sampling pipeline

void Reset()

Create a copy of this sampling pipeline

ISamplingPipeline Clone()