Class NpzFile
A lazily-loaded .npz archive — NumPy's NpzFile.
public sealed class NpzFile : IReadOnlyDictionary<string, NDArray>, IReadOnlyCollection<KeyValuePair<string, NDArray>>, IEnumerable<KeyValuePair<string, NDArray>>, IEnumerable, IDisposable
- Inheritance
-
NpzFile
- Implements
- Inherited Members
Remarks
A .npz is a ZIP archive of .npy members. Nothing is decoded until a key is
accessed, and each array is cached from then on, so a huge archive costs only what is read
out of it.
Keys work with or without the .npy suffix — npz["weights"] and
npz["weights.npy"] are the same member — while files reports the stripped
names, matching NumPy.
The members are NumPy's, names included (API-parity batch 13): files, f,
zip, fid, allow_pickle, max_header_size,
keys(), values(), items(), get(string, NDArray) and close(). The
dictionary interface's Count / Keys / Values are explicit (NumPy has len(z),
keys() and values()); ContainsKey(string) and TryGetValue(string, out NDArray) stay the C#
spellings of NumPy's key in z, and foreach keeps the dictionary's name / array pairs.
A missing key raises NumPy's errors: the indexer a KeyError ('x is not a file in the
archive') and dot access an AttributeError (x). Footgun: although the class is an
IReadOnlyDictionary<TKey, TValue>, its indexer does NOT throw .NET's
KeyNotFoundException; test with ContainsKey(string) or TryGetValue(string, out NDArray) first, or
catch KeyError.
The archive holds an open file handle, so dispose it:
using var npz = np.load_npz("model.npz");
NDArray w = npz["weights"];
NDArray b = npz.f.biases; // dot access, NumPy's BagObj
Any number of threads may read members of one archive at the same time (NumPy's NpzFile is not
thread-safe; this one is). A member already loaded is returned without a lock. Everything that reads the
archive itself (loading a member, GetRawBytes(string), the magic check of IsArray(string)) runs
under one gate, because the archive reads a single underlying stream, so new members load one at a time.
Two threads asking for the same member first get the same instance. Dispose() waits for a load in
progress, and every access after it raises ObjectDisposedException. zip hands out
the archive itself and bypasses the gate.
Constructors
NpzFile(Stream, bool, bool, long)
Open an archive over a stream - NumPy's NpzFile(fid, own_fid=False, allow_pickle=False, *,
max_header_size=10000).
public NpzFile(Stream fid, bool own_fid = false, bool allow_pickle = false, long max_header_size = 10000)
Parameters
fidStreamA readable, seekable stream holding the ZIP archive (NumPy's file object).
own_fidboolWhen true, closing the archive also closes
fid, and fid reports it (NumPy keepsself.fidonly for an owned file).allow_pickleboolWhether members are trusted; see ReadArray(Stream, bool, long).
max_header_sizelongPer-member header size cap.
Remarks
NumPy's names since API-parity batch 13 (they were stream, ownStream,
allowPickle, maxHeaderSize); NumPy's pickle_kwargs has no NumSharp counterpart (its
unpickler takes no Python keyword dictionary).
Exceptions
- ArgumentNullException
fidis null.- InvalidDataException
The stream is not a ZIP archive (an owned stream is closed first).
NpzFile(string, bool, long)
Open an archive from a file path (NumPy's NpzFile(fid) with a path, which zipfile opens). The
file handle is owned: fid reports it and close() closes it.
public NpzFile(string fid, bool allow_pickle = false, long max_header_size = 10000)
Parameters
fidstringThe archive's path.
allow_pickleboolWhether members are trusted; see ReadArray(Stream, bool, long).
max_header_sizelongPer-member header size cap.
Exceptions
- ArgumentNullException
fidis null.- FileNotFoundException
No such file.
- InvalidDataException
The file is not a ZIP archive (the file is closed first).
Properties
this[string]
The array stored under key, with or without the .npy suffix.
Loaded on first access and cached; every later access, from any thread, returns the same instance.
public NDArray this[string key] { get; }
Parameters
keystringThe member name, with or without the
.npysuffix.
Property Value
- NDArray
The member's array, owned by the caller that first read it (the archive never disposes it).
Remarks
A cached member is returned without a lock. A member that is not cached yet is loaded under the archive's gate, so a thread loading one member waits for a thread loading another.
Exceptions
- ObjectDisposedException
The archive was disposed.
- KeyError
No such member: NumPy's
KeyError, whose text is Python'sstr()of it ('x is not a file in the archive', quoted).- FormatException
The member is not a .npy file — use GetRawBytes(string) — or its header is malformed.
- NotSupportedException
The member holds a dtype NumSharp cannot load (an object array, say).
- InvalidDataException
A compressed member's data is corrupt.
allow_pickle
Whether members are loaded as trusted input (NumPy's allow_pickle).
public bool allow_pickle { get; }
Property Value
f
Dot-notation access to members — NumPy's npz.f.weights (its BagObj). Requires a
dynamic receiver: NDArray w = npz.f.weights;. Null once close() ran, as NumPy's
self.f = None.
public dynamic f { get; }
Property Value
- dynamic
fid
The file NumPy's fid names: the stream this archive OWNS (a path's file, or a stream given with
own_fid: true), null for a borrowed stream (NumPy keeps self.fid only when it owns it) and after
close().
public Stream fid { get; }
Property Value
files
The array names in the archive, with .npy stripped — NumPy's .files (a repeated name
appears twice, as NumPy's list built from the zip's name list).
public IReadOnlyList<string> files { get; }
Property Value
max_header_size
The per-member header size cap (NumPy's max_header_size).
public long max_header_size { get; }
Property Value
zip
The underlying archive (NumPy's zip), for callers that need entry metadata; null once
close() ran, as NumPy's. Reading entries through it bypasses the gate that serializes this
class's own reads, so do not read it while other threads load members.
public ZipArchive zip { get; }
Property Value
- ZipArchive
The open archive, or null after close().
Methods
ContainsKey(string)
Whether the archive has this member (with or without the .npy suffix).
public bool ContainsKey(string key)
Parameters
keystring
Returns
Dispose()
Close the archive and release an owned file handle (close(), for using).
public void Dispose()
Remarks
Takes the archive's gate, so a member another thread is loading finishes first (a large member can make this wait), and that thread's next access raises ObjectDisposedException. The arrays already handed out stay valid: they belong to the callers that read them. Calling it again does nothing.
GetEnumerator()
Enumerate every member as a name/array pair, loading each in turn.
public IEnumerator<KeyValuePair<string, NDArray>> GetEnumerator()
Returns
GetRawBytes(string)
A member's raw bytes, whatever it holds. NumPy returns these from its indexer for
non-.npy members; for a .npy member this is the encoded file itself.
public byte[] GetRawBytes(string key)
Parameters
keystringThe member name, with or without the
.npysuffix.
Returns
- byte[]
A new array holding the member's bytes.
Remarks
Capped at ~2 GB by Array itself. A member larger than that can still be read as an array through the indexer, which streams instead of buffering. Reads the archive under its gate, like a load.
Exceptions
- ObjectDisposedException
The archive was disposed.
- KeyError
No such member (the indexer's NumPy
KeyErrorand text).- InvalidDataException
A compressed member's data is corrupt.
IsArray(string)
Whether key names a member that holds a .npy array.
public bool IsArray(string key)
Parameters
keystringThe member name, with or without the
.npysuffix.
Returns
- bool
True when the member is cached or starts with the .npy magic string; false when there is no such member or it is not a .npy file.
Remarks
A cached member answers without a lock; otherwise the magic check reads the archive under its gate.
Exceptions
- ObjectDisposedException
The archive was disposed.
ToString()
Formatted as NumPy's repr: NpzFile 'model.npz' with keys: a, b, c.
public override string ToString()
Returns
TryGetValue(string, out NDArray)
The array under key, or false if there is no such member.
public bool TryGetValue(string key, out NDArray value)
Parameters
Returns
close()
public void close()
get(string, NDArray)
The array under key, or default when there is no such member —
NumPy's get(key, default=None) (the Mapping method: no KeyError).
public NDArray get(string key, NDArray @default = null)
Parameters
keystringThe member name, with or without the
.npysuffix.defaultNDArrayThe answer for a missing member; null (NumPy's None) by default. A C# number converts.
Returns
- NDArray
The member's array (the indexer's instance), or
default.
Exceptions
- ObjectDisposedException
The archive was disposed.
- FormatException
The member is not a .npy file.
items()
Every member as a name / array pair — NumPy's items(). Enumerating it loads and caches every
member.
public IEnumerable<(string key, NDArray value)> items()
Returns
- IEnumerable<(string key, NDArray value)>
The pairs, in the archive's order (
foreach (var (name, array) in z.items())).
Exceptions
- ObjectDisposedException
The archive was disposed (checked as each member is read).
- FormatException
A member is not a .npy file.
keys()
The member names — NumPy's keys(), the same names as files.
public IEnumerable<string> keys()
Returns
- IEnumerable<string>
The names, in the archive's order.
Exceptions
- ObjectDisposedException
The archive was disposed.
values()
Every member's array — NumPy's values(). Enumerating it loads and caches every member.
public IEnumerable<NDArray> values()
Returns
- IEnumerable<NDArray>
The arrays, in the archive's order (each the instance the indexer returns).
Exceptions
- ObjectDisposedException
The archive was disposed (checked as each member is read).
- FormatException
A member is not a .npy file.