Skip to content

LLamaNativeBatch

Namespace: LLama.Native

Input data for llama_decode A llama_batch object can contain input about one or many sequences The provided arrays (i.e. token, embd, pos, etc.) must have size of n_tokens

1
public struct LLamaNativeBatch

Inheritance Object → ValueType → LLamaNativeBatch

Fields

n_tokens

The number of items pointed at by pos, seq_id and logits.

1
public int n_tokens;

tokens

Either n_tokens of llama_token, or NULL, depending on how this batch was created

1
public LLamaToken* tokens;

embd

Either n_tokens * embd * sizeof(float) or NULL, depending on how this batch was created

1
public Single* embd;

pos

the positions of the respective token in the sequence

1
public LLamaPos* pos;

n_seq_id

https://github.com/ggerganov/llama.cpp/blob/master/llama.h#L139 ???

1
public Int32* n_seq_id;

seq_id

the sequence to which the respective token belongs

1
public LLamaSeqId** seq_id;

logits

if zero, the logits for the respective token will not be output

1
public Byte* logits;