DefaultSamplingPipeline
Namespace: LLama.Sampling
An implementation of ISamplePipeline which mimics the default llama.cpp sampling
public sealed class DefaultSamplingPipeline : BaseSamplingPipeline, ISamplingPipeline, System.IDisposable
Inheritance Object → BaseSamplingPipeline → DefaultSamplingPipeline
Implements ISamplingPipeline, IDisposable
Properties
LogitBias
Bias values to add to certain logits
public Dictionary<int, float> LogitBias { get; }
Property Value
RepeatPenalty
Repetition penalty, as described in https://arxiv.org/abs/1909.05858
public float RepeatPenalty { get; set; }
Property Value
AlphaFrequency
Frequency penalty as described by OpenAI: https://platform.openai.com/docs/api-reference/chat/create
Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text
so far, decreasing the model's likelihood to repeat the same line verbatim.
public float AlphaFrequency { get; set; }
Property Value
AlphaPresence
Presence penalty as described by OpenAI: https://platform.openai.com/docs/api-reference/chat/create
Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the
text so far, increasing the model's likelihood to talk about new topics.
public float AlphaPresence { get; set; }
Property Value
Temperature
Temperature to apply (higher temperature is more "creative")
public float Temperature { get; set; }
Property Value
TopK
Number of tokens to keep in TopK sampling
public int TopK { get; set; }
Property Value
TailFreeZ
Z value for tail free sampling
public float TailFreeZ { get; set; }
Property Value
TypicalP
P value for locally typical sampling
public float TypicalP { get; set; }
Property Value
TopP
P value for TopP sampling
public float TopP { get; set; }
Property Value
MinP
P value for MinP sampling
public float MinP { get; set; }
Property Value
PenalizeNewline
Whether the newline value should be protected from being modified by logit bias and repeat penalty
public bool PenalizeNewline { get; set; }
Property Value
Grammar
Grammar to constrain valid tokens
public SafeLLamaGrammarHandle Grammar { get; set; }
Property Value
Constructors
DefaultSamplingPipeline()
public DefaultSamplingPipeline()
Methods
ProcessLogits(SafeLLamaContextHandle, Span<Single>, ReadOnlySpan<LLamaToken>)
protected void ProcessLogits(SafeLLamaContextHandle ctx, Span<float> logits, ReadOnlySpan<LLamaToken> lastTokens)
Parameters
logits Span<Single>
lastTokens ReadOnlySpan<LLamaToken>
ProcessTokenDataArray(SafeLLamaContextHandle, LLamaTokenDataArray, ReadOnlySpan<LLamaToken>)
protected LLamaToken ProcessTokenDataArray(SafeLLamaContextHandle ctx, LLamaTokenDataArray candidates, ReadOnlySpan<LLamaToken> lastTokens)
Parameters
candidates LLamaTokenDataArray
lastTokens ReadOnlySpan<LLamaToken>
Returns
Accept(SafeLLamaContextHandle, LLamaToken)
public void Accept(SafeLLamaContextHandle ctx, LLamaToken token)
Parameters
token LLamaToken
Clone()
public ISamplingPipeline Clone()