NativeLibraryConfig
Namespace: LLama.Native
Allows configuration of the native llama.cpp libraries to load and use. All configuration must be done before using any other LLamaSharp methods!
1 | |
Inheritance Object → NativeLibraryConfig
Properties
Instance
Get the config instance
1 | |
Property Value
LibraryHasLoaded
Check if the native library has already been loaded. Configuration cannot be modified if this is true.
1 | |
Property Value
Methods
WithLibrary(String, String)
Load a specified native library as backend for LLamaSharp. When this method is called, all the other configurations will be ignored.
1 | |
Parameters
llamaPath String
The full path to the llama library to load.
llavaPath String
The full path to the llava library to load.
Returns
Exceptions
InvalidOperationException
Thrown if LibraryHasLoaded is true.
WithCuda(Boolean)
Configure whether to use cuda backend if possible.
1 | |
Parameters
enable Boolean
Returns
Exceptions
InvalidOperationException
Thrown if LibraryHasLoaded is true.
WithAvx(AvxLevel)
Configure the prefferred avx support level of the backend.
1 | |
Parameters
level AvxLevel
Returns
Exceptions
InvalidOperationException
Thrown if LibraryHasLoaded is true.
WithAutoFallback(Boolean)
Configure whether to allow fallback when there's no match for preferred settings.
1 | |
Parameters
enable Boolean
Returns
Exceptions
InvalidOperationException
Thrown if LibraryHasLoaded is true.
SkipCheck(Boolean)
Whether to skip the check when you don't allow fallback. This option may be useful under some complex conditions. For example, you're sure you have your cublas configured but LLamaSharp take it as invalid by mistake.
1 | |
Parameters
enable Boolean
Returns
Exceptions
InvalidOperationException
Thrown if LibraryHasLoaded is true.
WithLogs(Boolean)
Whether to output the logs to console when loading the native library with your configuration.
1 | |
Parameters
enable Boolean
Returns
Exceptions
InvalidOperationException
Thrown if LibraryHasLoaded is true.
WithLogs(LLamaLogLevel)
Enable console logging with the specified log logLevel.
1 | |
Parameters
logLevel LLamaLogLevel
Returns
Exceptions
InvalidOperationException
Thrown if LibraryHasLoaded is true.
WithSearchDirectories(IEnumerable<String>)
Add self-defined search directories. Note that the file stucture of the added directories must be the same as the default directory. Besides, the directory won't be used recursively.
1 | |
Parameters
directories IEnumerable<String>
Returns
WithSearchDirectory(String)
Add self-defined search directories. Note that the file stucture of the added directories must be the same as the default directory. Besides, the directory won't be used recursively.
1 | |
Parameters
directory String
Returns
CheckAndGatherDescription(LibraryName)
1 | |
Parameters
library LibraryName
Returns
AvxLevelToString(AvxLevel)
1 | |
Parameters
level AvxLevel