ICustomSampler
Namespace: LLama.Native
A custom sampler stage for modifying logits or selecting a token
1 | |
Implements IDisposable
Attributes NullableContextAttribute
Properties
Name
The human readable name of this stage
1 | |
Property Value
Methods
Apply(LLamaTokenDataArrayNative&)
Apply this stage to a set of logits. This can modify logits or select a token (or both). If logits are modified the Sorted flag must be set to false.
1 | |
Parameters
tokenData LLamaTokenDataArrayNative&
Remarks:
If the logits are no longer sorted after the custom sampler has run it is critically important to set Sorted=false. If unsure, always set it to false, this is a safe default.
Accept(LLamaToken)
Update the internal state of the sampler when a token is chosen
1 | |
Parameters
token LLamaToken
Reset()
Reset the internal state of this sampler
1 | |
Clone()
Create a clone of this sampler
1 | |