IInferenceParams

Namespace: LLama.Abstractions

The parameters used for inference.

public interface IInferenceParams

Properties

number of tokens to keep from initial prompt

public abstract int TokensKeep { get; set; }

how many new tokens to predict (n_predict), set to -1 to inifinitely generate response until it complete.

public abstract int MaxTokens { get; set; }

Sequences where the model will stop generating further tokens.

public abstract IReadOnlyList<string> AntiPrompts { get; set; }

Set a custom sampling pipeline to use.

public abstract ISamplingPipeline SamplingPipeline { get; set; }

If true, special characters will be converted to text. If false they will be invisible.

public abstract bool DecodeSpecialTokens { get; set; }