Skip to content

IModelParams

Namespace: LLama.Abstractions

The parameters for initializing a LLama model.

1
public interface IModelParams

Properties

MainGpu

main_gpu interpretation depends on split_mode: NoneThe GPU that is used for the entire mode.RowThe GPU that is used for small tensors and intermediate results.LayerIgnored.

1
public abstract int MainGpu { get; set; }

Property Value

Int32

SplitMode

How to split the model across multiple GPUs

1
public abstract GPUSplitMode SplitMode { get; }

Property Value

GPUSplitMode

GpuLayerCount

Number of layers to run in VRAM / GPU memory (n_gpu_layers)

1
public abstract int GpuLayerCount { get; }

Property Value

Int32

UseMemorymap

Use mmap for faster loads (use_mmap)

1
public abstract bool UseMemorymap { get; }

Property Value

Boolean

UseMemoryLock

Use mlock to keep model in memory (use_mlock)

1
public abstract bool UseMemoryLock { get; }

Property Value

Boolean

ModelPath

Model path (model)

1
public abstract string ModelPath { get; }

Property Value

String

TensorSplits

how split tensors should be distributed across GPUs

1
public abstract TensorSplitsCollection TensorSplits { get; }

Property Value

TensorSplitsCollection

VocabOnly

Load vocab only (no weights)

1
public abstract bool VocabOnly { get; }

Property Value

Boolean

LoraAdapters

List of LoRA adapters to apply

1
public abstract AdapterCollection LoraAdapters { get; }

Property Value

AdapterCollection

LoraBase

base model path for the lora adapter (lora_base)

1
public abstract string LoraBase { get; }

Property Value

String

MetadataOverrides

Override specific metadata items in the model

1
public abstract List<MetadataOverride> MetadataOverrides { get; }

Property Value

List<MetadataOverride>