Struct NDIterState
Core iterator state with dynamically allocated arrays for both dimensions and operands.
NUMSHARP DIVERGENCE: Unlike NumPy's fixed NPY_MAXDIMS=64 and NPY_MAXARGS=64, NumSharp supports unlimited dimensions AND unlimited operands. All arrays are allocated dynamically based on actual NDim and NOp values.
public struct NDIterState
- Inherited Members
Fields
ArrayWritebackPtrs
Each operand's ARRAY position at the start of the current buffer — NumPy's NIT_DATAPTRS while buffering:
the source of the fill and the destination of the write-back, and what the next fill of the reduce double loop
compares with to decide whether a buffer still holds its operand's data (NumPy's reuse rule). Size = NOp.
public long* ArrayWritebackPtrs
Field Value
- long*
BaseOffsets
Base offsets for each operand. Size = NOp.
public long* BaseOffsets
Field Value
- long*
BufIterEnd
The iteration index one past the current inner loop — NumPy's NBF_BUFITEREND. In the reduce double loop
it marks the end of the current CORE (not of the whole fill), and moves by BufTransferSize per
outer step.
public long BufIterEnd
Field Value
BufStrides
Each operand's inner-loop stride in BYTES — NumPy's NBF_STRIDES: the buffer element size for a buffered
operand (0 for a buffer holding a single repeated element, and for a reduce operand whose element repeats along
the core), the operand's own innermost array stride for a BUFNEVER one. Size = NOp.
public long* BufStrides
Field Value
- long*
BufTransferSize
The current inner loop's element count — NumPy's NBF_SIZE, what GetInnerLoopSizePtr exposes.
The whole fill outside the reduce double loop; one core inside it when the fill holds more than one core (the
fill is then BufTransferSize * ReduceOuterSize elements). Zero before the first fill and after the
iteration ended, which is what makes a second write-back of the same buffer a no-op.
public long BufTransferSize
Field Value
BufferSize
Elements one buffer fill may cover — NumPy's NBF_BUFFERSIZE. Holds the requested maximum until
NDIterBufferManager.FindBufferingSetup runs, then the setup's best_size: a whole number of
CoreSize cores at most the requested maximum, except that GROWINNER over operands that need no
buffer keeps the whole outer block (the inner loop may then exceed the requested size). Every buffer holds
this many elements.
public long BufferSize
Field Value
Buffers
Buffer pointers for each operand (null for a BUFNEVER operand, and while allocation is delayed). Size = NOp.
public long* Buffers
Field Value
- long*
Coords
Current coordinates. Dynamically allocated: size = NDim.
public long* Coords
Field Value
- long*
CoreOffset
The position inside its core of the iteration index the iterator was last moved to — NumPy's
coreoffset, set by every reposition (iterindex % CoreSize). A fill starting inside a core covers
only the rest of that core and never reuses a buffer.
public long CoreOffset
Field Value
CoreSize
The elements of one core — NumPy's NBF_CORESIZE: the product of the extents of the axes inside the outer
dimension. At least 1.
public long CoreSize
Field Value
DType
Legacy: primary dtype.
public NPTypeCode DType
Field Value
DataPtrs
Current data pointers for each operand. Size = NOp.
public long* DataPtrs
Field Value
- long*
ElementSizes
Element sizes for each operand (based on buffer dtype). Size = NOp.
public int* ElementSizes
Field Value
- int*
FlatIndex
Flat index for C_INDEX or F_INDEX tracking. Updated by Advance() when HASINDEX flag is set.
public long FlatIndex
Field Value
IndexReset
The tracked flat index at the iteration's coordinate origin (every Coords entry 0): NumPy's
reset value of the index "operand", which starts at 0 and moves to (shape - 1) * stride along every
axis the construction flipped. FlatIndex is IndexReset + Σ Coords[d] * IndexStrides[d].
Meaningful only when HASINDEX is set (set by NumSharp.Backends.Iteration.NDIterState.ComputeIndexStrides()).
public long IndexReset
Field Value
IndexStrides
Per iteration axis, the step of the tracked C or F flat index (NumPy keeps the index as one more "operand" whose strides count index units). Size = StridesNDim (the axes can only shrink). Computed by NumSharp.Backends.Iteration.NDIterState.ComputeIndexStrides() once the axes are ordered and flipped, then permuted and merged by coalescing exactly like an operand's strides — which is what lets an index-tracking iterator coalesce when the index allows it, as NumPy's does, without losing the index. Meaningful only when HASINDEX is set.
public long* IndexStrides
Field Value
- long*
InnerStrides
Inner strides for each operand (gathered from main Strides array for fast access). Layout: [op0_inner_stride, op1_inner_stride, ...] Size = NOp.
public long* InnerStrides
Field Value
- long*
IsCIndex
True if tracking C-order index, false for F-order. Only meaningful when HASINDEX flag is set.
public bool IsCIndex
Field Value
ItFlags
Iterator flags (NDIterFlags bitmask).
public uint ItFlags
Field Value
IterEnd
Range end for ranged iteration.
public long IterEnd
Field Value
IterIndex
Current iteration index.
public long IterIndex
Field Value
IterSize
Total number of iterations.
public long IterSize
Field Value
IterStart
Range start for ranged iteration.
public long IterStart
Field Value
MaskOp
Mask operand index (-1 if none).
public int MaskOp
Field Value
MaxBufferSize
The buffer size the iterator was built with (its buffersize argument, 0 resolved to NumPy's 8192), as
NDIterBufferManager.FindBufferingSetup received it. NumPy keeps no such field (it runs the setup once);
NumSharp needs it for the one place it runs the setup again — RemoveMultiIndex over a partly merged
buffered iteration, where NumPy's stale setup would walk wrong elements and can write outside an operand.
public long MaxBufferSize
Field Value
NDim
Number of dimensions after coalescing.
public int NDim
Field Value
NOp
Number of operands.
public int NOp
Field Value
OpDTypes
Buffer/target dtypes for each operand. Size = NOp.
public byte* OpDTypes
Field Value
- byte*
OpItFlags
Per-operand flags. Size = NOp.
public ushort* OpItFlags
Field Value
OpSrcDTypes
Source array dtypes for each operand (used for casting). Size = NOp.
public byte* OpSrcDTypes
Field Value
- byte*
OpStringKinds
The fixed-width string CLASS of each operand, one byte per operand: bit StringBufferUnicode is set
when the operand's ITERATION (buffer) descriptor is U, bit StringSourceUnicode when its ARRAY
descriptor is U; zero for a numeric operand and for S. A string type code names no dtype (both
classes share String, and the width lives on the descriptor), so this byte and the
element sizes are what a buffered string transfer reads: class and width on both sides. Size = NOp.
public byte* OpStringKinds
Field Value
- byte*
OuterAxisStrides
Each operand's array stride along the buffering setup's outer axis, in BYTES — NumPy's
NAD_STRIDES(outer_axisdata), which the fills, the write-backs and the reuse rule of the reduce double loop
read. Filled by the setup and kept across a remove_multi_index merge that leaves the setup in place (a
merge into one axis, or above the outer position, where the outer axis keeps these strides); a partial merge
below the outer position runs the setup again (NDIterBufferManager.RebuildBufferingSetup). Size = NOp.
public long* OuterAxisStrides
Field Value
- long*
OuterDim
The buffering setup's outer dimension in NumPy's numbering — the axis index counted from the INNERMOST axis
(NumPy's NBF_OUTERDIM), not a NumSharp axis index: NumSharp axis NDim - 1 - OuterDim. Kept in NumPy's
numbering because remove_multi_index coalesces the axes after the setup and NumPy keeps using that index
on the merged axes (see StaleOuterRemaining).
public int OuterDim
Field Value
Perm
Axis permutation (maps iterator axis to original axis). Dynamically allocated: size = NDim.
public sbyte* Perm
Field Value
ReduceOuterPtrs
Each operand's pointer at the start of the current outer step of the reduce double loop — NumPy's
NBF_REDUCE_OUTERPTRS: the buffer start (or the array position for a BUFNEVER operand) after a fill, then
advanced by ReduceOuterStrides at every outer step. Size = NOp.
public long* ReduceOuterPtrs
Field Value
- long*
ReduceOuterSize
The number of outer steps of the current fill — NumPy's NBF_REDUCE_OUTERSIZE: the fill's size divided by
CoreSize, rounded down. Zero for a fill shorter than one core (a fill that starts inside a core),
which NumPy's write-back then multiplies into a zero-element transfer.
public long ReduceOuterSize
Field Value
ReduceOuterStrides
Each operand's pointer step between the outer steps of the reduce double loop, in BYTES — NumPy's
NBF_REDUCE_OUTERSTRIDES: 0 for a reduce operand whose core repeats, the element size for a buffered reduce
operand whose element repeats along the core, a whole core for any other buffered operand, the array stride along
the outer axis for a BUFNEVER one. MinValue (NumPy's NPY_MIN_INTP, "invalid") when the
iteration does not use the reduce loop. Size = NOp.
public long* ReduceOuterStrides
Field Value
- long*
ReducePos
The current outer step of the reduce double loop, in [0, ReduceOuterSize) — NumPy's
NBF_REDUCE_POS. Zero outside the reduce loop. IsFirstVisit reads it: a reduce operand whose
outer stride is 0 repeats its core at every step after the first.
public long ReducePos
Field Value
ResetDataPtrs
Reset data pointers (base + offset). Size = NOp.
public long* ResetDataPtrs
Field Value
- long*
Shape
Shape after coalescing. Dynamically allocated: size = NDim.
public long* Shape
Field Value
- long*
SrcElementSizes
Source element sizes for each operand (based on source dtype). Size = NOp.
public int* SrcElementSizes
Field Value
- int*
StackAllocThreshold
Threshold for using stackalloc vs heap allocation for temporary buffers. Arrays with more dimensions than this will use heap allocation.
public const int StackAllocThreshold = 64
Field Value
StaleOuterRemaining
-1, or — after remove_multi_index merged every axis into one, the setup's outer axis with them, so
that OuterDim no longer names an axis — the remaining extent of the vanished outer axis at the merge
(its extent minus its coordinate), which NumPy then reads from that axis's stale storage at every fill: no
reposition updates it.
public long StaleOuterRemaining
Field Value
Strides
Strides for each operand along each axis. Dynamically allocated: size = NDim * NOp. Layout: [op0_axis0, op0_axis1, ..., op1_axis0, op1_axis1, ...] Access: Strides[operand * NDim + axis]
Note: Unlike fixed layout which uses MaxDims spacing, dynamic layout packs strides contiguously based on actual NDim.
public long* Strides
Field Value
- long*
StridesNDim
Allocated NDim for the Strides array. Used to compute correct offsets when NDim changes (e.g., after coalescing). Strides array maintains its original allocation size for safety.
public int StridesNDim
Field Value
StringBufferUnicode
OpStringKinds bit: the operand's iteration (buffer) descriptor is U (UCS-4 units).
public const byte StringBufferUnicode = 1
Field Value
StringBufferVString
OpStringKinds bit: the operand's iteration (buffer) descriptor is a NEP 55 StringDType. Its
16-byte cells look like an |S16 element to the type code and the element size, so this bit is what keeps a
buffer transfer from moving them as bytes (a cell means nothing without its buffer's allocator).
public const byte StringBufferVString = 4
Field Value
StringSourceUnicode
OpStringKinds bit: the operand array's own descriptor is U (UCS-4 units).
public const byte StringSourceUnicode = 2
Field Value
StringSourceVString
OpStringKinds bit: the operand array's own descriptor is a NEP 55 StringDType.
public const byte StringSourceVString = 8
Field Value
VStringContext
A GCHandle to the iterator's NumSharp.Backends.Iteration.NDIterVStringContext (the
descriptors and allocators of its NEP 55 StringDType operands), or zero when no operand has a
StringDType side. Released by FreeDimArrays(), the step every teardown of a state runs.
public nint VStringContext
Field Value
Properties
Flags
Legacy: flags (lower bits of ItFlags).
public NDIterFlags Flags { readonly get; set; }
Property Value
IsContiguousCopy
Check if this is a contiguous copy operation (legacy).
public readonly bool IsContiguousCopy { get; }
Property Value
Size
Legacy: total size (alias for IterSize).
public long Size { readonly get; set; }
Property Value
Methods
Advance()
Advance iterator by one position using ripple carry.
public void Advance()
AllocateDimArrays(int, int, int)
Allocate all dynamic arrays for given ndim and nop. Must be called before using any pointer fields. Initializes Perm to identity permutation [0, 1, 2, ...].
public void AllocateDimArrays(int ndim, int nop, int stridesNDim = -1)
Parameters
FreeDimArrays()
Free all dynamically allocated arrays. Must be called before freeing the state itself.
public void FreeDimArrays()
GetArrayWritebackPtr(int)
Get array writeback pointer for operand.
public void* GetArrayWritebackPtr(int op)
Parameters
opint
Returns
- void*
GetBufStride(int)
Get buffer stride for operand.
public long GetBufStride(int op)
Parameters
opint
Returns
GetBuffer(int)
Get buffer pointer for operand.
public void* GetBuffer(int op)
Parameters
opint
Returns
- void*
GetCoordsPointer()
Get pointer to Coords array.
public long* GetCoordsPointer()
Returns
- long*
GetDataPointer(int)
Get data pointer (legacy interface).
public readonly nint GetDataPointer(int operand)
Parameters
operandint
Returns
GetDataPtr(int)
Get current data pointer for operand.
public void* GetDataPtr(int op)
Parameters
opint
Returns
- void*
GetElementSize(int)
Get element size for operand.
public int GetElementSize(int op)
Parameters
opint
Returns
GetInnerStrideArray()
Get inner stride array pointer - returns contiguous array of inner strides for all operands. Layout: [op0_inner_stride, op1_inner_stride, ...]
public long* GetInnerStrideArray()
Returns
- long*
GetOpDType(int)
Get operand dtype.
public NPTypeCode GetOpDType(int op)
Parameters
opint
Returns
GetOpDescr(int)
The descriptor of operand op's ITERATION (buffer) side, rebuilt from its type code, element
size and string class — the state stores no descriptor objects, and a string's width and class are exactly the
two facts a type code drops. Byte order is not stored (buffers are native), so a non-native request reads back
native. A StringDType side is the descriptor instance the iterator's NumSharp.Backends.Iteration.NDIterVStringContext
keeps (its na_object and coerce are not facts of the state).
public DType GetOpDescr(int op)
Parameters
opintThe operand index.
Returns
- DType
A sized descriptor (for messages and debug output; not cached).
GetOpFlags(int)
Get operand flags.
public NDIterOpFlags GetOpFlags(int op)
Parameters
opint
Returns
GetOpSrcDType(int)
Get source array dtype for operand.
public NPTypeCode GetOpSrcDType(int op)
Parameters
opint
Returns
GetOpSrcDescr(int)
The descriptor of operand op's ARRAY side — the counterpart of GetOpDescr(int).
public DType GetOpSrcDescr(int op)
Parameters
opintThe operand index.
Returns
- DType
A sized descriptor (for messages and debug output; not cached).
GetReduceOuterPtr(int)
Get reduce outer pointer for operand.
public void* GetReduceOuterPtr(int op)
Parameters
opint
Returns
- void*
GetReduceOuterStride(int)
Get reduce outer stride for operand.
public long GetReduceOuterStride(int op)
Parameters
opint
Returns
GetResetDataPtr(int)
Get reset data pointer for operand.
public void* GetResetDataPtr(int op)
Parameters
opint
Returns
- void*
GetShapePointer()
Get pointer to Shape array.
public long* GetShapePointer()
Returns
- long*
GetSrcElementSize(int)
Get source element size for operand.
public int GetSrcElementSize(int op)
Parameters
opint
Returns
GetStride(int, int)
Get stride for operand at axis.
public long GetStride(int axis, int op)
Parameters
Returns
GetStridesPointer(int)
Get pointer to strides for a specific operand. Uses actual NDim (or StridesNDim if NDim changed after allocation).
public long* GetStridesPointer(int operand)
Parameters
operandint
Returns
- long*
GotoIterIndex(long)
Moves the position to iterindex: the coordinates (decomposed innermost axis first), the
tracked flat index, and every operand's ARRAY pointer from its reset pointer — NumPy's axis half of
npyiter_goto_iterindex.
public void GotoIterIndex(long iterindex)
Parameters
iterindexlongThe target iteration index (not range-checked: the callers check).
Remarks
The data pointers then point into the operands' arrays, never into buffers: a buffered iterator follows this with
NDIterBufferManager.CopyToBuffers (which records these array positions as the buffer's write-back targets
and switches the buffered operands' pointers to their buffers), and sets CoreOffset through
NDIterBufferManager.GotoIterIndex. The buffer-reuse marks are NOT cleared here: NumPy's reduce loop moves
the position between a write-back and the next fill, and the fill's reuse rule needs the marks of the previous one.
InitializeFlatIndex()
Initialize FlatIndex based on current coordinates. Should be called after HASINDEX flag is set and all axis setup is complete.
public void InitializeFlatIndex()
IsStringOp(int)
Whether either side of operand op (its array or its iteration dtype) is a fixed-width string.
public bool IsStringOp(int op)
Parameters
opintThe operand index.
Returns
- bool
True when a buffer transfer of this operand must take the string route.
IsVStringOp(int)
Whether either side of operand op is a NEP 55 StringDType — its transfers then go through
the iterator's NumSharp.Backends.Iteration.NDIterVStringContext (each string duplicated into the destination's allocator), never
as bytes.
public bool IsVStringOp(int op)
Parameters
opintThe operand index.
Returns
- bool
True for a
StringDTypearray or iteration side.
NeedsCast(int)
Check if operand needs casting (source dtype != buffer dtype). Two string sides share one type code, so for
them the width and the class decide: <U3 into a <U5 buffer, or |S3 into <U3, is a
cast (NumPy's string_to_string zero-pad copy / its S ↔ U casts).
public bool NeedsCast(int op)
Parameters
opint
Returns
Reset()
Moves the position to IterStart (which may be above 0 for a ranged iterator): the coordinates,
the tracked flat index and the ARRAY pointers — the axis half of NumPy's NpyIter_Reset
(npyiter_goto_iterindex(ITERSTART)).
public void Reset()
Remarks
A buffered iterator's buffers are not touched: NDIterRef.Reset writes the current buffer back first and
fills the next one after, as NumPy's reset does. The buffer-reuse marks (BUF_REUSABLE)
are left alone too: the next fill decides reuse itself, from the previous buffer's start positions.
SetArrayWritebackPtr(int, void*)
Set array writeback pointer for operand.
public void SetArrayWritebackPtr(int op, void* ptr)
Parameters
SetBufStride(int, long)
Set buffer stride for operand.
public void SetBufStride(int op, long stride)
Parameters
SetBuffer(int, void*)
Set buffer pointer for operand.
public void SetBuffer(int op, void* ptr)
Parameters
SetDataPointer(int, nint)
Set data pointer (legacy interface).
public void SetDataPointer(int operand, nint pointer)
Parameters
SetDataPtr(int, void*)
Set current data pointer for operand.
public void SetDataPtr(int op, void* ptr)
Parameters
SetOpDType(int, DType)
Sets an operand's iteration (buffer) dtype from its DESCRIPTOR: the type code, the element size in bytes, and
for a fixed-width string the class bit of OpStringKinds. The only way to give a string operand
its element size: String has none of its own (<U5 is 20 bytes, |S5 5).
public void SetOpDType(int op, DType descr)
Parameters
Exceptions
- NotSupportedException
descrhas no storage lane (a datetime descriptor before Stage C).
SetOpDType(int, NPTypeCode)
Set operand dtype (buffer/target dtype).
public void SetOpDType(int op, NPTypeCode dtype)
Parameters
opintdtypeNPTypeCode
SetOpFlags(int, NDIterOpFlags)
Set operand flags.
public void SetOpFlags(int op, NDIterOpFlags flags)
Parameters
opintflagsNDIterOpFlags
SetOpSrcDType(int, DType)
Sets an operand's SOURCE (array) dtype from its descriptor — the counterpart of SetOpDType(int, DType) for OpSrcDTypes, SrcElementSizes and the source class bit of OpStringKinds.
public void SetOpSrcDType(int op, DType descr)
Parameters
Exceptions
- NotSupportedException
descrhas no storage lane.
SetOpSrcDType(int, NPTypeCode)
Set source array dtype for operand.
public void SetOpSrcDType(int op, NPTypeCode dtype)
Parameters
opintdtypeNPTypeCode
SetReduceOuterPtr(int, void*)
Set reduce outer pointer for operand.
public void SetReduceOuterPtr(int op, void* ptr)
Parameters
SetReduceOuterStride(int, long)
Set reduce outer stride for operand.
public void SetReduceOuterStride(int op, long stride)
Parameters
SetResetDataPtr(int, void*)
Set reset data pointer for operand.
public void SetResetDataPtr(int op, void* ptr)
Parameters
SetStride(int, int, long)
Set stride for operand at axis.
public void SetStride(int axis, int op, long value)
Parameters
UpdateInnerStrides()
Update the InnerStrides array from the main Strides array. Must be called after coalescing or axis removal.
public void UpdateInnerStrides()