NativeLibraryConfigContainer
Namespace: LLama.Native
A class to set same configurations to multiple libraries at the same time.
1 | |
Inheritance Object → NativeLibraryConfigContainer
Attributes NullableContextAttribute, NullableAttribute
Methods
ForEach(Action<NativeLibraryConfig>)
Do an action for all the configs in this container.
1 | |
Parameters
action Action<NativeLibraryConfig>
WithLibrary(String, String)
Load a specified native library as backend for LLamaSharp. When this method is called, all the other configurations will be ignored.
1 | |
Parameters
llamaPath String
The full path to the llama library to load.
llavaPath String
The full path to the llava library to load.
Returns
Exceptions
InvalidOperationException
Thrown if LibraryHasLoaded is true.
WithCuda(Boolean)
Configure whether to use cuda backend if possible.
1 | |
Parameters
enable Boolean
Returns
Exceptions
InvalidOperationException
Thrown if LibraryHasLoaded is true.
WithVulkan(Boolean)
Configure whether to use vulkan backend if possible.
1 | |
Parameters
enable Boolean
Returns
Exceptions
InvalidOperationException
Thrown if LibraryHasLoaded is true.
WithAvx(AvxLevel)
Configure the prefferred avx support level of the backend.
1 | |
Parameters
level AvxLevel
Returns
Exceptions
InvalidOperationException
Thrown if LibraryHasLoaded is true.
WithAutoFallback(Boolean)
Configure whether to allow fallback when there's no match for preferred settings.
1 | |
Parameters
enable Boolean
Returns
Exceptions
InvalidOperationException
Thrown if LibraryHasLoaded is true.
SkipCheck(Boolean)
Whether to skip the check when you don't allow fallback. This option may be useful under some complex conditions. For example, you're sure you have your cublas configured but LLamaSharp take it as invalid by mistake.
1 | |
Parameters
enable Boolean
Returns
Exceptions
InvalidOperationException
Thrown if LibraryHasLoaded is true.
WithSearchDirectories(IEnumerable<String>)
Add self-defined search directories. Note that the file structure of the added directories must be the same as the default directory. Besides, the directory won't be used recursively.
1 | |
Parameters
directories IEnumerable<String>
Returns
WithSearchDirectory(String)
Add self-defined search directories. Note that the file structure of the added directories must be the same as the default directory. Besides, the directory won't be used recursively.
1 | |
Parameters
directory String
Returns
WithSelectingPolicy(INativeLibrarySelectingPolicy)
Set the policy which decides how to select the desired native libraries and order them by priority. By default we use DefaultNativeLibrarySelectingPolicy.
1 | |
Parameters
policy INativeLibrarySelectingPolicy
Returns
WithLogCallback(LLamaLogCallback)
Set the log callback that will be used for all llama.cpp log messages
1 | |
Parameters
callback LLamaLogCallback
Returns
Exceptions
WithLogCallback(ILogger)
Set the log callback that will be used for all llama.cpp log messages
1 | |
Parameters
logger ILogger
Returns
Exceptions
DryRun(INativeLibrary&, INativeLibrary&)
Try to load the native library with the current configurations, but do not actually set it to NativeApi.
You can still modify the configuration after this calling but only before any call from NativeApi.
1 | |
Parameters
loadedLLamaNativeLibrary INativeLibrary&
loadedLLavaNativeLibrary INativeLibrary&
Returns
Boolean
Whether the running is successful.