StatelessExecutor

Namespace: LLama

This executor infer the input as one-time job. Previous inputs won't impact on the response to current input.

public class StatelessExecutor : LLama.Abstractions.ILLamaExecutor

Properties

public bool IsMultiModal { get; }

public LLavaWeights ClipModel { get; }

public List<Byte[]> Images { get; }

The context used by the executor when running the inference.

public LLamaContext Context { get; private set; }

If true, applies the default template to the prompt as defined in the rules for llama_chat_apply_template template.

public bool ApplyTemplate { get; set; }

The system message to use with the prompt. Only used when StatelessExecutor.ApplyTemplate is true.

public string SystemMessage { get; set; }

Create a new stateless executor which will use the given model

public StatelessExecutor(LLamaWeights weights, IContextParams params, ILogger logger)

logger ILogger

public IAsyncEnumerable<string> InferAsync(string prompt, IInferenceParams inferenceParams, CancellationToken cancellationToken)

prompt String

inferenceParams IInferenceParams

cancellationToken CancellationToken