NativeLibraryConfig
Namespace: LLama.Native
Allows configuration of the native llama.cpp libraries to load and use. All configuration must be done before using any other LLamaSharp methods!
public sealed class NativeLibraryConfig
Inheritance Object → NativeLibraryConfig
Properties
Instance
Get the config instance
public static NativeLibraryConfig Instance { get; }
Property Value
LibraryHasLoaded
Check if the native library has already been loaded. Configuration cannot be modified if this is true.
public static bool LibraryHasLoaded { get; internal set; }
Property Value
Methods
WithLibrary(String, String)
Load a specified native library as backend for LLamaSharp. When this method is called, all the other configurations will be ignored.
public NativeLibraryConfig WithLibrary(string llamaPath, string llavaPath)
Parameters
llamaPath String
The full path to the llama library to load.
llavaPath String
The full path to the llava library to load.
Returns
Exceptions
InvalidOperationException
Thrown if LibraryHasLoaded is true.
WithCuda(Boolean)
Configure whether to use cuda backend if possible.
public NativeLibraryConfig WithCuda(bool enable)
Parameters
enable Boolean
Returns
Exceptions
InvalidOperationException
Thrown if LibraryHasLoaded is true.
WithAvx(AvxLevel)
Configure the prefferred avx support level of the backend.
public NativeLibraryConfig WithAvx(AvxLevel level)
Parameters
level AvxLevel
Returns
Exceptions
InvalidOperationException
Thrown if LibraryHasLoaded is true.
WithAutoFallback(Boolean)
Configure whether to allow fallback when there's no match for preferred settings.
public NativeLibraryConfig WithAutoFallback(bool enable)
Parameters
enable Boolean
Returns
Exceptions
InvalidOperationException
Thrown if LibraryHasLoaded is true.
SkipCheck(Boolean)
Whether to skip the check when you don't allow fallback. This option may be useful under some complex conditions. For example, you're sure you have your cublas configured but LLamaSharp take it as invalid by mistake.
public NativeLibraryConfig SkipCheck(bool enable)
Parameters
enable Boolean
Returns
Exceptions
InvalidOperationException
Thrown if LibraryHasLoaded is true.
WithLogs(Boolean)
Whether to output the logs to console when loading the native library with your configuration.
public NativeLibraryConfig WithLogs(bool enable)
Parameters
enable Boolean
Returns
Exceptions
InvalidOperationException
Thrown if LibraryHasLoaded is true.
WithLogs(LLamaLogLevel)
Enable console logging with the specified log logLevel.
public NativeLibraryConfig WithLogs(LLamaLogLevel logLevel)
Parameters
logLevel LLamaLogLevel
Returns
Exceptions
InvalidOperationException
Thrown if LibraryHasLoaded is true.
WithSearchDirectories(IEnumerable<String>)
Add self-defined search directories. Note that the file stucture of the added directories must be the same as the default directory. Besides, the directory won't be used recursively.
public NativeLibraryConfig WithSearchDirectories(IEnumerable<string> directories)
Parameters
directories IEnumerable<String>
Returns
WithSearchDirectory(String)
Add self-defined search directories. Note that the file stucture of the added directories must be the same as the default directory. Besides, the directory won't be used recursively.
public NativeLibraryConfig WithSearchDirectory(string directory)
Parameters
directory String
Returns
CheckAndGatherDescription(LibraryName)
internal static Description CheckAndGatherDescription(LibraryName library)
Parameters
library LibraryName
Returns
AvxLevelToString(AvxLevel)
internal static string AvxLevelToString(AvxLevel level)
Parameters
level AvxLevel