Skip to content

DefaultSamplingPipeline

Namespace: LLama.Sampling

An implementation of ISamplePipeline which mimics the default llama.cpp sampling

1
public sealed class DefaultSamplingPipeline : BaseSamplingPipeline, ISamplingPipeline, System.IDisposable

Inheritance Object → BaseSamplingPipeline → DefaultSamplingPipeline
Implements ISamplingPipeline, IDisposable

Properties

LogitBias

Bias values to add to certain logits

1
public Dictionary<int, float> LogitBias { get; }

Property Value

Dictionary<Int32, Single>

RepeatPenalty

Repetition penalty, as described in https://arxiv.org/abs/1909.05858

1
public float RepeatPenalty { get; set; }

Property Value

Single

AlphaFrequency

Frequency penalty as described by OpenAI: https://platform.openai.com/docs/api-reference/chat/create
Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim.

1
public float AlphaFrequency { get; set; }

Property Value

Single

AlphaPresence

Presence penalty as described by OpenAI: https://platform.openai.com/docs/api-reference/chat/create
Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.

1
public float AlphaPresence { get; set; }

Property Value

Single

Temperature

Temperature to apply (higher temperature is more "creative")

1
public float Temperature { get; set; }

Property Value

Single

TopK

Number of tokens to keep in TopK sampling

1
public int TopK { get; set; }

Property Value

Int32

TailFreeZ

Z value for tail free sampling

1
public float TailFreeZ { get; set; }

Property Value

Single

TypicalP

P value for locally typical sampling

1
public float TypicalP { get; set; }

Property Value

Single

TopP

P value for TopP sampling

1
public float TopP { get; set; }

Property Value

Single

MinP

P value for MinP sampling

1
public float MinP { get; set; }

Property Value

Single

PenalizeNewline

Whether the newline value should be protected from being modified by logit bias and repeat penalty

1
public bool PenalizeNewline { get; set; }

Property Value

Boolean

Grammar

Grammar to constrain valid tokens

1
public SafeLLamaGrammarHandle Grammar { get; set; }

Property Value

SafeLLamaGrammarHandle

Constructors

DefaultSamplingPipeline()

1
public DefaultSamplingPipeline()

Methods

ProcessLogits(SafeLLamaContextHandle, Span<Single>, ReadOnlySpan<LLamaToken>)

1
protected void ProcessLogits(SafeLLamaContextHandle ctx, Span<float> logits, ReadOnlySpan<LLamaToken> lastTokens)

Parameters

ctx SafeLLamaContextHandle

logits Span<Single>

lastTokens ReadOnlySpan<LLamaToken>

ProcessTokenDataArray(SafeLLamaContextHandle, LLamaTokenDataArray, ReadOnlySpan<LLamaToken>)

1
protected LLamaToken ProcessTokenDataArray(SafeLLamaContextHandle ctx, LLamaTokenDataArray candidates, ReadOnlySpan<LLamaToken> lastTokens)

Parameters

ctx SafeLLamaContextHandle

candidates LLamaTokenDataArray

lastTokens ReadOnlySpan<LLamaToken>

Returns

LLamaToken

Accept(SafeLLamaContextHandle, LLamaToken)

1
public void Accept(SafeLLamaContextHandle ctx, LLamaToken token)

Parameters

ctx SafeLLamaContextHandle

token LLamaToken

Clone()

1
public ISamplingPipeline Clone()

Returns

ISamplingPipeline