LLamaNativeBatch

Namespace: LLama.Native

Input data for llama_decode A llama_batch object can contain input about one or many sequences The provided arrays (i.e. token, embd, pos, etc.) must have size of n_tokens

public struct LLamaNativeBatch

Inheritance Object → ValueType → LLamaNativeBatch

Fields

n_tokens

The number of items pointed at by pos, seq_id and logits.

public int n_tokens;

tokens

Either n_tokens of llama_token, or NULL, depending on how this batch was created

public LLamaToken* tokens;

embd

Either n_tokens * embd * sizeof(float) or NULL, depending on how this batch was created

public Single* embd;

pos

the positions of the respective token in the sequence

public LLamaPos* pos;

n_seq_id

https://github.com/ggerganov/llama.cpp/blob/master/llama.h#L139 ???

public Int32* n_seq_id;

seq_id

the sequence to which the respective token belongs

public LLamaSeqId** seq_id;

logits

if zero, the logits for the respective token will not be output

public Byte* logits;