StatelessExecutor
Namespace: LLama
This executor infer the input as one-time job. Previous inputs won't impact on the response to current input.
public class StatelessExecutor : LLama.Abstractions.ILLamaExecutor
Inheritance Object → StatelessExecutor
Implements ILLamaExecutor
Properties
Model
The mode used by the executor when running the inference.
public LLamaModel Model { get; }
Property Value
Constructors
StatelessExecutor(LLamaModel)
public StatelessExecutor(LLamaModel model)
Parameters
model LLamaModel
The LLama model.
Methods
Infer(String, InferenceParams, CancellationToken)
public IEnumerable<string> Infer(string text, InferenceParams inferenceParams, CancellationToken cancellationToken)
Parameters
text String
inferenceParams InferenceParams
cancellationToken CancellationToken
Returns
InferAsync(String, InferenceParams, CancellationToken)
public IAsyncEnumerable<string> InferAsync(string text, InferenceParams inferenceParams, CancellationToken token)
Parameters
text String
inferenceParams InferenceParams
token CancellationToken