NativeLibraryConfig
Namespace: LLama.Native
Allows configuration of the native llama.cpp libraries to load and use. All configuration must be done before using any other LLamaSharp methods!
1 | |
Inheritance Object → NativeLibraryConfig
Attributes NullableContextAttribute, NullableAttribute
Properties
Instance
Caution
Please use NativeLibraryConfig.All instead, or set configurations for NativeLibraryConfig.LLama and NativeLibraryConfig.LLavaShared respectively.
Set configurations for all the native libraries, including LLama and LLava
1 | |
Property Value
All
Set configurations for all the native libraries, including LLama and LLava
1 | |
Property Value
LLama
Configuration for LLama native library
1 | |
Property Value
LLava
Configuration for LLava native library
1 | |
Property Value
LibraryHasLoaded
Check if the native library has already been loaded. Configuration cannot be modified if this is true.
1 | |
Property Value
Methods
WithLibrary(String)
Load a specified native library as backend for LLamaSharp. When this method is called, all the other configurations will be ignored.
1 | |
Parameters
libraryPath String
The full path to the native library to load.
Returns
Exceptions
InvalidOperationException
Thrown if LibraryHasLoaded is true.
WithCuda(Boolean)
Configure whether to use cuda backend if possible. Default is true.
1 | |
Parameters
enable Boolean
Returns
Exceptions
InvalidOperationException
Thrown if LibraryHasLoaded is true.
WithVulkan(Boolean)
Configure whether to use vulkan backend if possible. Default is true.
1 | |
Parameters
enable Boolean
Returns
Exceptions
InvalidOperationException
Thrown if LibraryHasLoaded is true.
WithAvx(AvxLevel)
Configure the prefferred avx support level of the backend. Default value is detected automatically due to your operating system.
1 | |
Parameters
level AvxLevel
Returns
Exceptions
InvalidOperationException
Thrown if LibraryHasLoaded is true.
WithAutoFallback(Boolean)
Configure whether to allow fallback when there's no match for preferred settings. Default is true.
1 | |
Parameters
enable Boolean
Returns
Exceptions
InvalidOperationException
Thrown if LibraryHasLoaded is true.
SkipCheck(Boolean)
Whether to skip the check when you don't allow fallback. This option may be useful under some complex conditions. For example, you're sure you have your cublas configured but LLamaSharp take it as invalid by mistake. Default is false;
1 | |
Parameters
enable Boolean
Returns
Exceptions
InvalidOperationException
Thrown if LibraryHasLoaded is true.
WithSearchDirectories(IEnumerable<String>)
Add self-defined search directories. Note that the file structure of the added directories must be the same as the default directory. Besides, the directory won't be used recursively.
1 | |
Parameters
directories IEnumerable<String>
Returns
WithSearchDirectory(String)
Add self-defined search directories. Note that the file structure of the added directories must be the same as the default directory. Besides, the directory won't be used recursively.
1 | |
Parameters
directory String
Returns
WithSelectingPolicy(INativeLibrarySelectingPolicy)
Set the policy which decides how to select the desired native libraries and order them by priority. By default we use DefaultNativeLibrarySelectingPolicy.
1 | |
Parameters
policy INativeLibrarySelectingPolicy
Returns
WithLogCallback(LLamaLogCallback)
Set the log callback that will be used for all llama.cpp log messages
1 | |
Parameters
callback LLamaLogCallback
Returns
Exceptions
WithLogCallback(ILogger)
Set the log callback that will be used for all llama.cpp log messages
1 | |
Parameters
logger ILogger
Returns
Exceptions
DryRun(INativeLibrary&)
Try to load the native library with the current configurations, but do not actually set it to NativeApi.
You can still modify the configuration after this calling but only before any call from NativeApi.
1 | |
Parameters
loadedLibrary INativeLibrary&
The loaded livrary. When the loading failed, this will be null.
However if you are using .NET standard2.0, this will never return null.
Returns
Boolean
Whether the running is successful.