StatelessExecutor

Namespace: LLama

This executor infer the input as one-time job. Previous inputs won't impact on the response to current input.

public class StatelessExecutor : LLama.Abstractions.ILLamaExecutor

Properties

The context used by the executor when running the inference.

public LLamaContext Context { get; private set; }

Create a new stateless executor which will use the given model

public StatelessExecutor(LLamaWeights weights, IModelParams params)

Use the constructor which automatically creates contexts using the LLamaWeights

Create a new stateless executor which will use the model used to create the given context

public StatelessExecutor(LLamaContext context)

public IEnumerable<string> Infer(string text, IInferenceParams inferenceParams, CancellationToken cancellationToken)

text String

inferenceParams IInferenceParams

cancellationToken CancellationToken

public IAsyncEnumerable<string> InferAsync(string text, IInferenceParams inferenceParams, CancellationToken cancellationToken)

text String

inferenceParams IInferenceParams

cancellationToken CancellationToken