Table of Contents

Strings, bytes and character arrays

NumSharp has NumPy's fixed-width Unicode (U), fixed-width bytes (S) and variable-width StringDType (T) dtypes. String arrays use the same indexing, views, broadcasting and file formats as other NDArrays. np.strings provides elementwise string operations, from concatenation and search to case conversion and encoding, without a Python runtime.

np.@char is the C# spelling of numpy.char, and NDCharArray is its chararray subclass. Neither is the same thing as NumSharp's existing Char dtype, which stores individual .NET UTF-16 code units. Use the table below to choose deliberately.

Choosing a text representation

What you need Use Important consequence
A fixed number of Unicode code points per element np.array(new[] { "cat", "bird" }, "U8") 32 bytes per element; longer writes truncate to 8 code points
Fixed-width byte strings / binary fields np.dtype("S8") 8 bytes per element; item() returns byte[]; conversion from Unicode uses ASCII
Text of varying lengths, optionally with missing values np.dtypes.StringDType() or np.dtype("T") No fixed text width; each element has a 16-byte cell plus any separately stored payload
NumPy's legacy whitespace-stripping character-array behavior np.@char.array(...) → NDCharArray S / U storage with different element-access and comparison rules
A .NET array of UTF-16 code units np.array(new[] { 'a', 'b' }) → Char 2 bytes per element, unsigned-integer behavior; not U1, S1 or StringDType
using NumSharp;

using var scope = NDScope.Open(); // Reclaim arrays created by this example when it ends.
var labels = np.array(new[] { "pear", "fig", "apple", "fig" }); // U5
var sorted = np.sort(labels);                                  // ['apple' 'fig' 'fig' 'pear']
var (values, counts) = np.unique(labels, return_counts: true);   // ['apple' 'fig' 'pear'], [1 2 1]
var repeated = np.strings.multiply(labels, 2);                 // ['pearpear' 'figfig' 'appleapple' 'figfig']

var nullable = np.dtypes.StringDType(na_object: null);
var notes = np.array(new object[] { "short", null }, nullable);
notes[0] = "This text can grow without a fixed-width truncation.";
var missing = notes.item(1);                                   // null, not the text "None"

Examples below show focused calls; use using or an NDScope around array work in an application. NDListArray split results and np.nditer iterators are disposable holders: dispose those explicitly, since an NDScope reclaims arrays, not holders.

Use np.strings for elementwise text operations and the usual np.* functions for sorting, searching, unique values, indexing and reshaping. np.@char provides NumPy's legacy string API and the NDCharArray class; its split functions return NDListArray, a container of per-element lists of pieces.


Fixed-width string dtypes

NumPy dtype Descriptor Bytes per character Example itemsize
str_ U, <U5 4 (one UCS-4 code point, native byte order) np.dtype("U5") 20
bytes_ S, \|S5 1 np.dtype("S5") 5

The width is part of the dtype. Every element of a <U5 array holds up to five characters:

  • A shorter value is padded with NUL characters.
  • Reading an element removes the trailing NULs. NULs inside the value stay.
  • A value that is too long is cut to the width when you write it, as NumPy does.
var names = np.array(new[] { "cat", "dog", "bird" });
names.dtype;             // dtype('<U4') - the longest value sets the width
names.dtype.itemsize;    // 16 - 4 characters x 4 bytes
np.dtype("S5").str;      // "|S5"
np.dtype("U5").name;     // "str160" - NumPy's name counts bits

The data buffer holds the characters directly, as in NumPy. A U character is a 4-byte code point. An S character is one byte:

np.array(new[] { "ab" }).tobytes();               // 61 00 00 00 62 00 00 00
np.array(new[] { "ab" }).astype("S3").tobytes();  // 61 62 00

A character outside the Basic Multilingual Plane (an emoji) is ONE U code point. C# stores it as two UTF-16 chars, but np.array(new[] { "😀" }) is <U1.


Creating string arrays

np.array(new[] { "cat", "dog" });                    // ['cat', 'dog'], <U3
np.array(new[] { "cat", "dog" }, np.dtype("S5"));    // [b'cat', b'dog'], |S5
np.zeros(new Shape(2), np.dtype("U3"));              // ['', ''], <U3
np.full(new Shape(2), "ab");                         // ['ab', 'ab'], <U2
NDArray.Scalar("abc");                               // 'abc', a 0-d <U3 array
np.frombuffer(Encoding.ASCII.GetBytes("abcdef"), "S3");   // [b'abc', b'def'], no copy

A C# string is one text value. A byte[] passed directly to np.array is a numeric uint8 array, not a byte-string scalar. Build an S array from byte-string values with an object[], or use NDArray.Scalar((object)bytes) for one S scalar:

np.array(new object[] { Encoding.ASCII.GetBytes("ab"), Encoding.ASCII.GetBytes("cde") });   // [b'ab', b'cde'], |S3

Footgun: np.array(new[] { bytes1, bytes2 }) with a byte[][] is a jagged NUMERIC array in C#. It becomes a 2-D uint8 array, not an S array.

NumPy's character dtype np.dtype("c") is an |S1 that keeps the type character 'c'. Creating an array from values with it splits every text into one byte per character, so equal-length rows make a 2-D byte map:

np.array(new[] { "SFFF", "FHFH" }, np.dtype("c"));   // [[b'S' b'F' b'F' b'F'] [b'F' b'H' b'F' b'H']], (2, 4) |S1
np.array("abc", np.dtype("c"));                      // [b'a' b'b' b'c'], (3,)
np.array(new[] { "ab", "abc" }, np.dtype("c"));      // ValueError: ... inhomogeneous shape after 1 dimensions ...

A cast never splits: astype("c") and np.asarray(existingArray, "c") keep each value's first byte, as S1 does. Pass the C# string[] to np.array, not np.asarray, which converts it to a <U array before it reads the dtype.


Reading and writing elements

names.item(1);                     // "dog" - a C# string
names.astype("S4").item(1);        // byte[] { 100, 111, 103 } - b'dog'
NDArray.AsStringArray(names);      // { "cat", "dog", "bird" }
names[1];                          // array('dog', dtype='<U3') - NumPy's scalar np.str_('dog'), a copy as wide as its text
names[1] = "zebras";               // the array is now ['cat', 'zebr', 'bird']: cut to 4 characters
names[2] = null;                   // NumPy's None is stored as its str(): ['cat', 'zebr', 'None']

GetString, SetString, GetStringAt and SetStringAt belong to the Char dtype: they read a vector of UTF-16 chars as one string. They refuse a U or S array with an InvalidOperationException.


Dtype integration

String descriptors take part in promotion and in the casting rules exactly as in NumPy:

np.promote_types(np.dtype("S3"), np.dtype("U2"));           // dtype('<U3')
np.result_type(new[] { "U3", "S7" });                        // dtype('<U7')
np.can_cast(np.dtype("S3"), np.dtype("U3"), "safe");         // true - ASCII bytes fit in Unicode
np.can_cast(np.dtype("U3"), np.dtype("S3"), "same_kind");    // false
np.can_cast(np.dtype("U5"), np.dtype("U3"), "safe");         // false - the cast would cut values

Moving data

Views, copies and element movement work on string arrays as on numeric ones: slicing, transposes, reshapes, copy, concatenate, take, boolean masks, where, pad, copyto, put and the other indexing routines.

np.pad pads text in every mode that moves elements: constant (with text, or a number written as text), edge, reflect, symmetric, wrap and empty. maximum and minimum pad a StringDType array with its extremes; NumPy has no such loop for S / U and raises. mean, median and linear_ramp compute with numbers and raise NumPy's errors for text.

Casts between string dtypes follow NumPy:

  • A narrower width cuts values.
  • U to S encodes ASCII.
  • S to U decodes ASCII.
np.concatenate(new[] { np.array(new[] { "a" }), np.array(new[] { "long" }) });   // ['a', 'long'], <U4
names.astype("U2");                        // ['ca', 'do', 'bi']
np.array(new[] { "é" }).astype("S1");      // UnicodeEncodeError

The last call raises UnicodeEncodeError with Python's message: 'ascii' codec can't encode character '\xe9' in position 0: ordinal not in range(128). Its encoding, @object, start, end and reason are Python's: ascii, é, 0, 1, ordinal not in range(128).


Converting between text and numbers

Numbers to text

A number becomes the text of its NumPy scalar's str(). An unsized "U" or "S" request takes the width NumPy reserves for the number type: 5 for bool, 21 for int64, 32 for the floats, 64 for complex. A sized request cuts the text to its width:

np.arange(3).astype("U");                          // ['0' '1' '2'], <U21
np.array(new[] { 1.5, 1e16, 1e-5 }).astype("U");   // ['1.5' '1e+16' '1e-05'], <U32
np.array(new[] { true, false }).astype("U");       // ['True' 'False'], <U5
np.array(new[] { 0.1f }).astype("U");              // ['0.1'] - float32 prints its own shortest digits
np.array(new[] { 3.14159 }).astype("U3");          // ['3.1']

The same text is written wherever a number goes into a text array: an element, fill, np.full, np.ones, np.pad, np.copyto, and a masked array's fill value:

var u = np.array(new[] { "ab", "cd" });
u[0] = 5;                                     // ['5' 'cd']
u[1] = 2.5;                                   // ['5' '2.'] - cut to the width
np.full(new Shape(2), 7, "U");                // ['7' '7'], <U1
np.ones(new Shape(2), "U");                   // ['1' '1'], <U1
np.pad(np.array(new[] { "ab", "c" }), 1);     // ['0' 'ab' 'c' '0']

Text to numbers

A cast to a number parses each element. int() takes decimal digits with an optional sign, underscores between digits and surrounding whitespace. float() also takes inf, nan and exponents. complex() takes 1+2j forms, with or without parentheses. An element that does not parse raises NumPy's ValueError, and an integer that does not fit raises NumPy's OverflowError:

np.array(new[] { "1", " 22 ", "-3", "1_000" }).astype(np.int64);    // [   1   22   -3 1000]
np.array(new[] { "1.5", "inf", "-nan", "1e3" }).astype(np.float64); // [   1.5    inf    nan 1000. ]
np.array(new[] { "1+2j", "3", "(4-5j)" }).astype(np.complex128);    // [1.+2.j 3.+0.j 4.-5.j]
np.array(new[] { "1", "x" }).astype(np.int64);      // ValueError: invalid literal for int() with base 10: np.str_('x')
np.array(new[] { "1", "a" }).astype(np.float64);    // ValueError: could not convert string to float: np.str_('a')
np.array(new[] { "300" }).astype(np.int8);          // OverflowError: Python integer 300 out of bounds for int8

A bytes (S) element parses the same way. The error names the first element that fails, in NumPy's order.

A C# string written into a numeric array is NumPy's Python str. It parses the same way, and its error quotes the string itself:

var a = np.zeros(new Shape(3), np.int32);
a[0] = "12";        // [12  0  0]
a[0] = "hello";     // ValueError: invalid literal for int() with base 10: 'hello'

Conversions inside other calls

NumPy converts between text and numbers inside other calls too, and NumSharp follows:

np.add(np.array(new[] { 1.5, 2.0 }), np.array(new long[] { 1, 2 }), dtype: "U");   // ['1.51' '2.02'], <U53
np.equal(names, np.array(new[] { "cat", "cow", "bird" }), np.empty(new Shape(3), "U5"));   // ['True' 'False' 'True']
np.add(np.array(new[] { 1.0, 2.5 }), np.array(new[] { 1.0, 3.0 }), np.zeros(new Shape(2), "U5"));   // ['2.0' '5.5']
np.prod(np.array(new[] { "1.5", "2", "4" }), (int?)null, np.float64);   // 12.0 - the text is parsed first
  • np.add with a string dtype= writes each number as text, then concatenates.
  • A ufunc with a text out= writes each result as text, cut to the width of out.
  • A reduction with a numeric dtype= parses the text first.
  • A reduction or accumulation into a text out= resolves its loop as NumPy does. np.cumsum of U text into a StringDType out concatenates (['c' 'ca' 'cab'] for ['c', 'a', 'b']). Integer counts reduced by prod into a text out repeat the first count's text. A numeric loop parses a text input and writes its running values into a text out. np.median and np.percentile into a text out leave NumPy's partial result there before NumPy's refusal.

With where=, NumPy also parses the CURRENT text of out into the loop's number type before it computes, so a masked-off 'x' in a float loop raises ValueError.

Breaking change: a C# string converts to a 0-d U array, as np.array(str) does in NumPy. NDArray x = "[1, 2, 3]" is array('[1, 2, 3]', dtype='<U9'). It no longer parses a numeric list. astype parses an individual numeric text (np.array("1.5").astype(np.float64)), not list syntax. For delimited numbers use np.fromstring("1, 2, 3", sep: ",") without the brackets.


Printing

ToString() is NumPy's str(), and ToString(true) is NumPy's repr(), including Python's quoting rules:

names.ToString();                                     // ['cat' 'dog' 'bird']
names.ToString(true);                                 // array(['cat', 'dog', 'bird'], dtype='<U4')
np.array2string(np.array(new[] { "a'b", "c\"d" }));   // ["a'b" 'c"d']

Computing on text

Comparisons

==, !=, <, <=, >, >= and the ufuncs np.equal … np.greater_equal compare two text arrays of one class (U with U, S with S), of any widths, as NumPy does. Strings compare code unit by code unit as unsigned values. Trailing NULs never count ('a' == 'a\0'), embedded NULs do. Broadcasting, every memory layout, out=, where= and dtype=bool work as for numbers. A C# string against a text array is NumPy's str scalar:

names == names;                       // [ True  True  True]
names < np.array(new[] { "dog" });    // [ True False  True]
names > "cat";                        // [False  True False]

The names np.strings.equal, not_equal, less, less_equal, greater and greater_equal are the same comparison ufuncs; they do not strip whitespace.

A bytes array against a str array has no comparison loop in NumPy: == is all False, != all True, and < raises NumPy's TypeError.

Truth values

A string is True when it is not empty, that is when any code unit is not NUL. A string of spaces and the text 'False' are True. astype(bool), np.any, np.all, np.nonzero, np.count_nonzero, np.where, np.compress, np.logical_and / np.logical_or / np.logical_xor, np.place and np.putmask masks, and np.trim_zeros read it. np.logical_not has no text loop, as in NumPy; invert an explicitly computed Boolean array instead:

np.array(new[] { "", "a", " " }).astype(np.@bool);   // [False  True  True]
np.count_nonzero(names);                            // 3

Concatenation and repetition

np.add and the + operator concatenate two text arrays of one class, element by element. The result is as wide as the two widths together. Trailing NULs never count, embedded NULs do. A C# string is NumPy's str scalar. A string on the left stays the first operand. np.strings.add is np.add:

names + "s";                                      // ['cats' 'dogs' 'birds'], <U5
"a " + names;                                     // ['a cat' 'a dog' 'a bird'], <U6
np.add(names, np.array(new[] { "!", "?", "" }));  // ['cat!' 'dog?' 'bird'], <U5

np.strings.multiply repeats each string. The result is as wide as the longest repetition. A count less than 1 gives an empty string:

np.strings.multiply(names, 2);                                  // ['catcat' 'dogdog' 'birdbird'], <U8
np.strings.multiply(names, np.array(new long[] { 1, 0, -1 }));  // ['cat' '' ''], <U3

The np.multiply ufunc cannot size its result, so it needs out=. It writes each repetition cut to the width of out. Without out=, np.multiply(names, 2) and names * 2 raise NumPy's TypeError, as in NumPy:

np.multiply(names, 2, np.zeros(new Shape(3L), "U5"));   // ['catca' 'dogdo' 'birdb']

Broadcasting, every memory layout, out= (a narrower or wider text array receives the result cut or padded) and where= work as for numbers. A bytes array with a str array has no loop: np.add raises NumPy's UFuncBinaryResolutionError (a TypeError). A repetition longer than 2^63 - 1 characters raises NumPy's OverflowError.

np.strings applies the familiar string operations element by element. String arguments and bounds broadcast against the input, so each element can have its own search text, start and end:

var words = np.array(new[] { "cat", "dog" });
np.strings.str_len(words);                          // [3 3]
np.strings.find(words, "a");                        // [1 -1]
np.strings.startswith(words, "c");                  // [True False]
np.strings.count(np.array(new[] { "banana", "pear" }), "a"); // [3 1]
Function Result
str_len(a) Length of each element, as int64
find(a, sub, start, end) / rfind First / last matching position, or -1
index(a, sub, start, end) / rindex Matching position; raises ValueError("substring not found") if absent
count(a, sub, start, end) Number of non-overlapping matches
startswith(a, prefix, start, end) / endswith Boolean array

Positions count bytes for S, Unicode code points for U and StringDType. For example, str_len of "é中😀" is 3 for either Unicode dtype, not its UTF-16 or UTF-8 byte count. Fixed-width trailing NUL padding does not contribute to the length. An S search takes an S argument; a C# string is U, so use a byte-string scalar rather than mixing the two classes.

Stripping, padding, replacing and slicing

Function Behavior
strip, lstrip, rstrip Remove whitespace or a supplied set of characters from both ends / the left / the right
replace(a, old, new, count) Replace occurrences, optionally limiting their number
center, ljust, rjust Pad to a width with fillchar
zfill(a, width) Pad with zeros after a leading sign
expandtabs(a, tabsize) Expand tabs at tab stops
slice(a, start, stop, step) Slice the characters inside each element, not the array's axes
np.strings.strip(np.array(new[] { " cat ", "dog\t" })); // ['cat' 'dog']
np.strings.replace(words, "a", "oo");                 // ['coot' 'dog']
np.strings.center(words, 7, fillchar: "-");            // ['--cat--' '--dog--']
np.strings.zfill(np.array(new[] { "-7", "12" }), 4);   // ['-007' '0012']
np.strings.slice(words, 0, 2);                         // ['ca' 'do']

The output-width rule belongs to the function. Stripping and character slicing keep a fixed-width input's dtype; padding and replacement size their output from the resulting text. StringDType results need no fixed width. An array slice such as words["0:1"] selects elements and can share memory; np.strings.slice transforms the text and produces a new result.

strip uses ASCII whitespace for S and Python's Unicode whitespace for U and StringDType. This differs from the legacy np.@char comparisons, which strip only ASCII whitespace even for a U operand.

Partitioning each string

np.strings.partition(a, sep) returns three arrays: text before the first separator, the separator itself, and text after it. rpartition uses the last separator. When the separator is absent, partition puts the original text in the first part; rpartition puts it in the last.

var fields = np.array(new[] { "a-b", "c" });
var parts = np.strings.partition(fields, "-"); // ['a' 'c'], ['-' ''], ['b' '']
np.@char.partition(fields, "-");              // [['a' '-' 'b'], ['c' '' '']]

The np.@char forms stack the three parts on a new last axis, so their shape is the broadcast input shape followed by 3. NDCharArray.partition(sep) follows this string operation, not the sorting operation np.partition(a, kth).

For S / U, NumSharp returns three arrays where NumPy returns field views of one structured array; values match, but ownership and strides differ. A StringDType input requires a StringDType separator too: create one with np.array("-", np.dtype("T")). A plain C# string is a U separator and has no matching T partition loop in NumPy.

Unicode properties and case conversion

isalpha, isalnum, isdigit, isdecimal, isnumeric, isspace, islower, isupper and istitle return Boolean arrays. Byte strings use ASCII rules; isdecimal and isnumeric accept U and StringDType, not S. Unicode properties and full case mappings follow CPython 3.12's Unicode 15.0.0 database, rather than the machine's culture or the installed .NET Unicode version.

var numbers = np.array(new[] { "12", "²", "½" });
np.strings.isdecimal(numbers); // [True False False]
np.strings.isdigit(numbers);   // [True True False]
np.strings.isnumeric(numbers); // [True True True]

upper, lower, swapcase, capitalize and title perform Python's case conversion. A mapping may expand one code point into several. Fixed-width results keep the input dtype, so an expansion can be cut; use StringDType when it must fit in full:

np.strings.upper(np.array(new[] { "ßa" }));                     // ['SS'], U2
np.strings.upper(np.array(new[] { "ßa" }, np.dtype("T")));     // ['SSA'], StringDType()

translate applies a translation table and also keeps the input dtype. A Unicode table maps code points to replacements or deletion; the bytes form uses a 256-byte table and optional deletechars. deletechars is ignored for Unicode, as in NumPy.

Formatting, encoding and decoding

mod(a, values) applies Python's % formatting to each element; the values broadcast against the format strings. encode converts Unicode to byte strings, and decode converts byte strings to Unicode:

np.strings.mod("item-%02d", np.array(new long[] { 1, 12 })); // ['item-01' 'item-12']
var encoded = np.strings.encode(np.array(new[] { "café" }), "utf-8"); // S5: 63 61 66 C3 A9
np.strings.decode(encoded, "utf-8");                       // ['café'], U4

Unlike astype("S"), which encodes ASCII, encode takes an explicit codec. Supported codecs include UTF-8, UTF-16, UTF-32, ASCII, Latin-1 and the single-byte code pages listed under Text encodings. The default is UTF-8 with errors: "strict". Python's ignore, replace, backslashreplace, xmlcharrefreplace, namereplace, surrogateescape and surrogatepass handlers are available where the codec and direction allow them. An unknown codec raises LookupError; an unsupported codec raises NotSupportedException.

Formatting accepts scalar or array values, not an object array of per-element tuples or dictionaries. A tuple of values is not automatically one formatting tuple per element. Fixed-width results of mod, encode and decode are sized to the longest result, rather than the input width.

Sorting

np.sort, np.argsort, ndarray.sort, np.partition and np.argpartition sort S and U text as NumPy does: by code point (U) or byte value (S), compared unsigned, a value before every longer value that starts with it. Every memory layout and every axis works, and axis: null sorts the flattened array:

var words = np.array(new[] { "pear", "fig", "apple", "fig" });
np.sort(words);                                         // ['apple' 'fig' 'fig' 'pear']
np.argsort(words);                                      // [2 1 3 0]
np.sort(np.array(new[] { "b", "A", "a", "é", "Z" }));  // ['A' 'Z' 'a' 'b' 'é'] - code-point order
words.sort();                                           // in place

NumPy's default sort is an introsort, which is not stable: the order of equal strings depends on the whole sort. NumSharp reproduces that order, so np.argsort returns NumPy's indices for every kind. kind: "stable" (or stable: true) keeps equal strings in their input order:

var grades = np.array("babacabcabacbacbabac".Select(c => c.ToString()).ToArray());
np.argsort(grades);                   // [ 1  3 16  5 13  8 18 10  0 17 15 12  9  6  2  7 14  4 11 19]
np.argsort(grades, kind: "stable");   // [ 1  3  5  8 10 13 16 18  0  2  6  9 12 15 17  4  7 11 14 19]

descending: true sorts the largest text first (NumPy 2.5). The default kind runs NumPy's introsort in reverse, whose order of equal strings NumSharp reproduces too; the stable kind keeps equal strings in their input order:

np.sort(words, descending: true);                     // ['pear' 'fig' 'fig' 'apple']
np.argsort(grades, descending: true);                 // [19 14  4 11  7 17 15 12  0  6  2  9 18 10  5 13  3 16  1  8]
np.argsort(grades, stable: true, descending: true);   // [ 4  7 11 14 19  0  2  6  9 12 15 17  1  3  5  8 10 13 16 18]

The keywords follow NumPy's rules:

  • kind is read by its first letter, in either case: quicksort (the default), heapsort (the default kind: NumPy clears its bit before it looks the sort up), mergesort and stable. Any other value is NumPy's ValueError.
  • kind cannot be given together with stable or descending, false included.
  • order is refused, because no NumSharp dtype has fields.
  • NumPy has no partition for text and sorts it instead, so np.partition returns the sorted text and np.argpartition the argsort indices.
np.sort(words, kind: "timsort");             // ValueError: sort kind must be one of 'quick', 'heap', or 'stable' (got 'timsort')
np.sort(words, kind: "stable", stable: true); // ValueError: `kind` and keyword parameters can't be provided at the same time. ...

Measured against NumPy 2.4.2 on the same machine (pinned to two cores): over fresh inputs of 1,000 strings every np.sort, np.argsort, np.partition and np.argpartition measured ran at least 1.58 times as fast as NumPy (3 times in the geometric mean), and over 100,000 and 1,000,000 strings they ran 1.7 to 19 times as fast. The slowest cases are argsorts of text outside Latin-1 (1.7 to 2 times NumPy's speed), whose characters take more room in the sort key, and argsorts of text with many repeated values (about twice NumPy's speed): NumSharp reproduces NumPy's order of the equal strings by replaying its introsort.

Searching

np.argmax, np.argmin, np.nanargmax, np.nanargmin (and the ndarray methods), np.searchsorted (and ndarray.searchsorted) and np.digitize search S and U text with NumPy's results. They compare as the sorts do: by code point (U) or byte value (S), unsigned.

var words = np.array(new[] { "pear", "fig", "apple", "fig" });
np.argmax(words);                                              // 0 ('pear')
np.argmin(words);                                              // 2 ('apple')
np.argmax(np.array(new[,] { { "b", "a" }, { "a", "c" } }), 1);  // [0 1]
var sorted = np.sort(words);                                   // ['apple' 'fig' 'fig' 'pear']
np.searchsorted(sorted, np.array(new[] { "fig", "kiwi" }));            // [1 3]
np.searchsorted(sorted, np.array(new[] { "fig", "kiwi" }), "right");   // [3 3]

The arg-reductions follow NumPy's rules:

  • Of several equal extremes, the first wins.
  • keepdims keeps the reduced axis.
  • out= takes an integer array NumPy accepts: bool through uint32 and the signed types. uint64 and floats raise its TypeError.
  • np.nanargmax and np.nanargmin of text are np.argmax and np.argmin: text has no NaN.

np.searchsorted first converts the keys and the haystack to their common string dtype, as NumPy does. A number is written as text, so np.searchsorted(np.array(new[] { "1", "10", "2", "5" }), 3) searches for "3" and returns 3. Bytes beside str are decoded.

Over an unsorted haystack, NumPy's answer comes from its exact sequence of comparisons, and NumSharp follows that sequence. np.digitize of text searches the bins converted to text, so the bins follow text order, not number order:

np.digitize(np.array(new[] { "10", "3", "25" }), np.array(new[] { 5, 20 }));   // [0 2 2]

Measured against NumPy 2.4.2 on the same machine (pinned to two cores, fresh inputs every call): the arg-reductions ran 3.4 times as fast as NumPy in the geometric mean, and np.searchsorted 4.6 times as fast (6 to 10 times over sorted haystacks of 100,000 strings, 1.4 to 4 times over unsorted ones, whose searches follow NumPy's exact sequence of comparisons). Two kinds of call stay close to NumPy's speed, because both libraries spend their time waiting on memory rather than comparing:

  • An arg-reduction over 100,000 fresh strings (1.1 to 1.5 times NumPy's speed). It reads every cache line of the array, at the memory's bandwidth.
  • A few keys searched in a long haystack (100 keys in 100,000 strings, 0.7 to 1.4 times NumPy's speed). Each step of a binary search reads an element whose address depends on the previous comparison, so the search waits on memory at every step.

Unique values, set routines and statistics

np.unique (every return_* flag, axis, equal_nan and sorted), np.unique_values, np.unique_counts, np.unique_inverse, np.unique_all, np.isin, np.intersect1d, np.union1d, np.setdiff1d, np.setxor1d, np.lexsort, np.sort_complex, np.histogram, np.histogram_bin_edges and the selection methods of np.quantile and np.percentile work on S and U text with NumPy's results. They order text as the sorts do.

var fruit = np.array(new[] { "pear", "fig", "apple", "fig", "pear" });
var (values, counts) = np.unique(fruit, return_counts: true);   // ['apple' 'fig' 'pear'], [1 2 2]
np.isin(fruit, np.array(new[] { "fig", "kiwi" }));               // [False  True False  True False]
np.intersect1d(fruit, np.array(new[] { "pear", "plum" }));       // ['pear']
np.setdiff1d(fruit, np.array(new[] { "pear" }));                 // ['apple' 'fig']
np.quantile(fruit, 0.5, method: "lower");                        // 'fig'
np.quantile(fruit, 0.5);   // TypeError: unsupported operand type(s) for -: 'numpy.str_' and 'numpy.str_'

Some results follow from how NumPy computes them:

  • Numbers and bytes beside text. np.isin and the set routines join their two operands into one array, as NumPy does. Numbers are written as text, and bytes are decoded as ASCII (a byte past ASCII raises NumPy's UnicodeDecodeError). With few test values, np.isin compares the elements instead, and text never equals a number there. NumPy switches between the two at 10 * len(ar1) ** 0.145 test values:

    np.isin(np.array(new[] { "1", "2" }), np.arange(20));            // [ True  True]: 20 values are joined
    np.isin(np.array(new[] { "1", "2" }), np.array(new long[] { 1 })); // [False False]: 1 value is compared
    
  • The order of np.unique_values. NumPy returns the distinct strings in the order of its hash table, which depends on the C++ library NumPy was built with. NumSharp returns them sorted; the values are the same. np.unique(..., sorted: false) without a return_* flag differs the same way.

  • np.sort_complex sorts the text first and parses it afterwards, so the result follows the text's order: np.sort_complex(np.array(new[] { "10", "9", "2+1j" })) is [10.+0.j 2.+1.j 9.+0.j].

  • np.histogram of text needs explicit edges, as text or as numbers. Number edges are compared as text, so a count can come out negative, as in NumPy: np.histogram(np.array(new[] { "1", "5", "10", "30", "2" }), np.array(new long[] { 0, 5, 10 })) counts [ 4 -2] ("10" sorts before "5" as text).

  • np.quantile and np.percentile compute the methods that pick an element: lower, higher, nearest, inverted_cdf, closest_observation, and linear with an integer q (np.quantile(fruit, 1) is 'pear'). The other methods interpolate, which subtracts text, so NumPy raises a TypeError and NumSharp raises the same one. np.percentile divides q by 100 first, so its linear method always interpolates.

  • Weighted quantiles pick an element without interpolation, using only method: "inverted_cdf". Weights must be non-negative and broadcast according to NumPy's reduced-axis rules. The selected value keeps the data's dtype:

    var fruit = np.array(new[] { "pear", "fig", "apple" });
    np.quantile(fruit, 0.5, method: "inverted_cdf", weights: np.array(new[] { 1.0, 3.0, 1.0 })); // 'fig'
    

Measured against NumPy 2.4.2 on the same machine (pinned to two cores, fresh inputs every call, 1,000 and 100,000 strings): every call measured ran at least 1.5 times as fast as NumPy, 4.2 times in the geometric mean. np.unique of text with few distinct values hashes the strings and sorts only the distinct ones, as NumPy does; past 1 in 16 distinct strings it sorts every string, which is faster for mostly distinct text (11 times NumPy's speed on 100,000 random strings). The slowest cases, 1.5 to 2 times NumPy's speed, are weighted histograms (each block's weights are summed in NumPy's tie order) and np.unique along an axis of small arrays.

The numpy.char module

np.@char is NumPy's numpy.char module. char is a C# keyword, so NumPy's np.char.equal(a, b) is np.@char.equal(a, b). It exposes the string operations above, except slice, with these additional conventions:

  • equal, not_equal, less, less_equal, greater and greater_equal compare two text arrays of one class after they remove the trailing NULs and ASCII whitespace (' ', '\t', '\n', '\v', '\f', '\r') of each string. Other whitespace ('\xa0', U+3000) and leading spaces stay.
  • compare_chararrays(a1, a2, cmp, rstrip) runs the comparison cmp ("==", "!=", "<", "<=", ">" or ">=", as a string or a byte[]), with the strip when rstrip is true. Any other cmp raises NumPy's TypeError (a bytes-like object is required, not 'int').
  • add is np.add.
  • multiply is np.strings.multiply, but a count that is not an integer gives NumPy's ValueError("Can only multiply by integers").
  • partition and rpartition return one array with a last axis of length 3, rather than three arrays.
  • array, asarray and chararray construct NDCharArray.
  • split, rsplit, splitlines and join compute over text elements; see Splitting and joining.
var padded = np.array(new[] { "cat  ", "dog\t", "bird" });
np.@char.equal(padded, "cat");                            // [ True False False]
np.equal(padded, "cat");                                  // [False False False]
np.@char.compare_chararrays(padded, "dog", "<", true);    // [ True False  True]
np.@char.add(padded, "!");                                // ['cat  !' 'dog\t!' 'bird!'], <U6
np.@char.multiply(np.array(new[] { "ab", "c" }), 3);      // ['ababab' 'ccc'], <U6
np.@char.multiply(padded, 2.5);                           // ValueError: Can only multiply by integers

The array arguments are NumPy's array_likes: an NDArray, a C# string (NumPy's str), a byte[] (NumPy's bytes in this module), a string[] or a byte[][] (a list of bytes). Two strings give a 0-d result. For the stripping comparisons, an S array against a U array is NumPy's NotImplemented, returned as null, and StringDType is refused with TypeError: comparison of non-string arrays. That restriction is not module-wide: add, multiply and the split/join family also accept StringDType.

np.@char also converts to the Char dtype, its meaning before the module existed, so astype(np.@char) and a.dtype == np.@char still work. Read a descriptor member through a cast: ((DType)np.@char).itemsize.

Measured against NumPy 2.4.2 (pinned to two cores, 1,000 to 1,000,000 strings): the comparisons ran 2.6 to 10 times as fast, np.@char.add 4.9 to 6.9 times, np.@char.multiply 1.5 to 5 times, np.@char.partition and rpartition 1.9 to 4.1 times. The slowest multiply case is a fresh 96 MB result, where both libraries pay the same page faults.

The chararray class

np.@char.chararray, np.@char.array and np.@char.asarray make NumPy's chararray: an NDCharArray, which is an NDArray of S or U text. A chararray differs from a plain string array in three ways:

  • Element access removes the trailing whitespace of the element, as Python's str.rstrip() does for U text (29 whitespace characters: the ASCII ones, '\x1c' to '\x1f', '\x85', '\xa0', U+3000 and the other Unicode spaces) and bytes.rstrip() does for S text (the 6 ASCII whitespace bytes). item(), tolist(), take(i) and the flat iterator keep the whitespace.
  • Views and copies of a chararray are chararrays: T, reshape, ravel, copy, astype, take, repeat and the other members it shares with NDArray.
  • A chararray holds text. A member that would make a chararray of numbers raises ValueError: Can only create a chararray from string data. NumPy's rule also allows int8 data and NumSharp's Char dtype.
var c = np.@char.array(new[] { "cat  ", "dog\t", "bird" });
c[0].item();            // "cat": a read-only 0-d chararray stands for Python's str
c.item(0);              // "cat  "
foreach (var s in c)    // "cat", "dog", "bird"
    Console.WriteLine(s);
c.ToString(true);       // chararray(['cat', 'dog', 'bird'], dtype='<U5')
c.T;                    // a chararray view
c.astype(np.float64);   // ValueError: Can only create a chararray from string data.

np.@char.array takes NumPy's arguments:

  • A string fills a chararray of len // itemsize elements. A byte[] is a read-only buffer of S text.
  • A list, or an S / U array, becomes a chararray view, a cast to another width or class, or a copy.
  • A number array, a StringDType array and a C# number become text the way NumPy's tolist() route writes them: np.@char.array(np.array(new[] { 0.1f })) is [b'0.10000000149011612'].

np.@char.asarray is array with copy: false, and returns an S chararray itself. a.view(typeof(NDCharArray)) makes a chararray view of an S / U array, as NumPy's a.view(np.char.chararray) does.

One difference from NumPy: with order: and text data, NumPy's np.char.array returns a plain array, because its np.asarray call drops the class. NumSharp returns a chararray with the same data and layout.

String methods on a chararray

A chararray has the corresponding methods, so c.upper(), c.strip() and c.find("a") operate on its stored text. These methods do not first strip the input through c[i]. Their return types follow NumPy:

Methods Result
capitalize, center, expandtabs, ljust, lower, partition, rjust, rpartition, swapcase, title, translate, upper, zfill NDCharArray; U results are copied by NumPy's chararray conversion, S results can be views of the computed array
lstrip, rstrip, strip, replace A chararray view of the text result
encode, decode, join A plain text NDArray
count, find, rfind, index, rindex, startswith, endswith, and the character predicates A plain int64 or Boolean NDArray
split, rsplit, splitlines NDListArray

The % operator applies np.@char.mod and returns a chararray. As with other NDArray operators, C# compound assignment rebinds the variable rather than performing NumPy's in-place update.

Operators and functions with a chararray

The comparison operators of a chararray remove the trailing whitespace before they compare, as np.@char.equal does. They do this also when a plain array is on the left, because Python asks the chararray first:

var c = np.@char.array(new[] { "ab  ", "b", "ab", "a " });
var u = np.array(new[] { "ab", "b", "ab  ", "a" });
c == "ab";      // [ True False  True False]
u == c;         // [ True  True  True  True]
c + "x";        // a chararray view, <U5: 'ab  x', 'bx', 'abx', 'a x'
c * 2;          // a chararray copy, <U8: 'ab  ab  ', 'bb', 'abab', 'a a '
c * 2.5;        // ValueError: Can only multiply by integers

An S chararray against U text is NumPy's NotImplemented. Then a plain array on the other side compares without the strip, or Python's fallback answers: == is False, != is True, and < raises TypeError: '<' not supported between instances of 'chararray' and 'bytes'.

The members:

  • sort() sorts in place. argsort() and searchsorted() return plain arrays.
  • argmax() and argmin() raise ValueError: Can only create a chararray from string data., because NumPy creates their index result as a chararray; giving them a plain out: allows the result. argpartition() has the same finalize error but no out parameter.
  • partition(sep) is the string partition of np.@char.partition, not the ordering operation np.partition(a, kth). Consequently, calls that pass an ordering axis keyword to the chararray method raise NumPy's TypeError: chararray.partition() got an unexpected keyword argument 'axis' (np.partition(c, 1) and unweighted quantile/median paths can reach it).

The np.* functions treat a chararray the way NumPy does:

  • The result is a chararray when NumPy calls a method of the array or views it with np.asanyarray: np.reshape, np.ravel, np.transpose, np.squeeze, np.sort, np.repeat, np.tile, np.flip, np.take, the split functions, np.zeros_like and the other *_like functions, np.copy(c, subok: true).
  • The result is a chararray view when it is the text result of a ufunc (np.add, np.strings.multiply) or of np.insert, np.delete, np.unique and the other functions that wrap their result. The view does not own its data.
  • The result is a plain array when NumPy converts with np.asarray: np.concatenate, np.stack, np.where, np.append, np.asarray, np.intersect1d, np.union1d and np.setxor1d. np.setdiff1d instead wraps its text result as a chararray view.
  • The call raises the ValueError when NumPy creates a non-text result as a chararray: np.argmax / np.argmin without out:, np.argpartition, and np.zeros_like(c, np.float64). A ufunc's preliminary cast can also trigger it: an unmasked logical ufunc copies a 0-d or short 1-d chararray into its bool loop dtype as a chararray (up to np.getbufsize(), normally 8,192 elements). This is not a general size limit on chararray operations.
np.sort(c);                                    // a chararray
np.concatenate(new NDArray[] { c, c });        // a plain array
np.argmax(c);                                  // ValueError: Can only create a chararray from string data.
np.unique(c, return_counts: true);             // values 'a ', 'ab', 'b' (a chararray), counts [1 2 1]
np.unique(c);                                  // 'a ', 'ab', 'ab  ', 'b': without a flag, no strip

The C# type of the variable decides which operators and members run, as for NDMatrix. A chararray in a variable of type NDArray uses the operators and members of NDArray: == does not strip, + returns a plain array, and argmax() returns the index. The np.* functions still find the chararray: np.reshape(plainVariable, ...) returns a chararray.

Other differences from NumPy:

  • c == null and c != null are reference tests that return a 0-d False and True. NumPy's c == None raises TypeError: comparison of non-string arrays.
  • np.squeeze of a chararray with no axis of length 1 returns a new chararray view. NumPy returns the same object.
  • A view of an element (np.atleast_2d(c[0])) is read-only, because the element is a read-only 0-d chararray. NumPy makes a writeable array from the Python str.
  • c + t with a StringDType array t of two or more dimensions raises NotSupportedException, because NumPy's result is a plain StringDType array and the + of NDCharArray returns a chararray. Call np.add(c, t) for NumPy's result.
  • A masked array of a chararray (np.ma.array(c)) holds the data as a plain array.

Measured against NumPy 2.4.2 (pinned to two cores, 1,000 to 1,000,000 strings): the comparison operators ran 2.8 to 6.1 times as fast, + and * 2.4 to 14 times, the np.* functions 1.3 to 9 times. The slowest cases are searchsorted (1.0 to 1.3 times NumPy's speed, the same as for a plain array: both libraries run one binary search per key), argmax with out: and a 1,000-string np.concatenate.

Splitting and joining: split, rsplit, splitlines and join

np.@char.split, rsplit and splitlines call Python's str or bytes method on every element, as NumPy does. The result is NumPy's object array of lists. NumSharp has no object dtype, so the result is an NDListArray: it has the shape of the input, and each element is the list of pieces of one string.

var words = np.array(new[] { "a b  c", "  lead trail  ", "" });
using NDListArray w = np.@char.split(words);    // [list(['a', 'b', 'c']) list(['lead', 'trail']) list([])]
np.@char.split(words, " ");                     // empty pieces where two separators touch
np.@char.rsplit(words, maxsplit: 1);            // ['a b', 'c'], ['  lead', 'trail'], []
np.@char.splitlines(np.array(new[] { "a\nb\r\nc" }), true);   // [list(['a\n', 'b\r\n', 'c'])]
w.item(0);                                      // a List<object>: "a", "b", "c"
w.tolist();                                     // the nested lists, every piece exact
w[0];                                           // the list of element 0: a 0-d list array, printed ['a', 'b', 'c']
w["1:"];                                        // an NDListArray view of the last two elements
w.ToString(true);                               // array([list(['a', 'b', 'c']), ...], dtype=object)

The rules are Python's:

  • Without a separator, the pieces are the runs of characters that are not whitespace: 29 whitespace characters for U and StringDType text, the 6 ASCII whitespace bytes for S text. With a separator, every occurrence splits, so empty pieces can occur. maxsplit limits the number of splits, from the end for rsplit.
  • splitlines breaks U text at \n, \r, \r\n, \v, \f, \x1c to \x1e, \x85, U+2028 and U+2029, and S text at \n, \r and \r\n only. keepends follows Python's truth rule: 0.5 and "x" are true, 0 and "" are false.
  • The separator, maxsplit and keepends can be arrays that broadcast against the text, one value per element.
  • bytes.split reads its separator as raw bytes, so a number works as a separator of S text: np.@char.split(s, 5) splits on the 8 bytes of the int64 value 5. A separator of U text must be a str or None.
  • When the separator is longer than a U string, the list holds the element itself, which NumPy prints as np.str_('...').

np.@char.join(sep, seq) joins the CHARACTERS of each string of seq with its separator, as Python's sep.join(seq) does for a string:

np.@char.join("-", "osd");                                    // 'o-s-d'
np.@char.join(new[] { "-", "." }, new[] { "ghc", "osd" });    // ['g-h-c', 'o.s.d'], <U5
np.@char.join("-", np.@char.split(words));                    // TypeError, as in NumPy 2.5.4 (see below)

The result has the class of seq: U or S as wide as the longest result, or StringDType(). An S sequence joins only when its strings are empty, because Python iterates bytes as numbers.

NumPy 2.5.4 cannot join the lists that split returns: it joins every list and then raises TypeError: currently only the no-argument instantiation is supported; use np.dtype instead. NumSharp raises the same error. Over an empty NDListArray NumPy returns an empty object array, which has no NDArray form: NumSharp raises NotSupportedException.

The errors are NumPy's, in NumPy's order. The first element that fails stops the call:

np.@char.split(words, "");            // ValueError: empty separator
np.@char.split(words, " ", 1.5);      // TypeError: 'numpy.float64' object cannot be interpreted as an integer
np.@char.split(words, new byte[] { 32 });   // TypeError: must be str or None, not numpy.bytes_
np.@char.split(np.array(new long[] { 1 }));  // TypeError: string operation on non-string array

In these four functions a C# null is Python's None: np.@char.split(null) raises NumPy's TypeError, and new object[] { " ", null } as a separator splits the second element at whitespace. The chararray members c.split(), c.rsplit(), c.splitlines() and c.join(seq) call the same functions on the chararray's elements with their trailing whitespace.

An NDListArray has the ndarray members of NumPy's object array, each with NumPy's parameters: the structural ones (T, reshape, ravel, copy, view, take, repeat, diagonal, squeeze, ...) and the ones that compare, sort, reduce, compute and assign lists with Python's list rules (==, <, sort, argsort, searchsorted, max, sum, cumsum, mean, var, clip, +, *, put, fill, ...).

w.sum();                                          // ['a', 'b', 'c', 'lead', 'trail']: the lists concatenated
w.cumsum(@out: np.@char.split(np.array(new[] { "x", "y", "z" })));   // into a list array of the same shape
w.reshape(new Shape(1, 3), 'C', copy: false);     // a view, or NumPy's ValueError when none exists
w.mean(dtype: np.@bool, where: np.array(new[] { true, false, true }));   // True: the masked truths over the count

A dtype= whose NumPy result is not a list raises NotSupportedException: w.sum(dtype: np.@bool) is NumPy's np.True_, which np.sum(w.astype(np.@bool), np.@bool) computes.

Differences from NumPy:

  • An element access (w[0], w[i, j]) returns a 0-d list array that stands for the list, where NumPy returns the Python list itself. w.item(i, j) returns the list as a List<object>.
  • Iterating a 1-D list array (foreach (NDArray pieces in w)) yields the pieces of each element as a new U or S array, where NumPy yields the lists.

Measured against NumPy 2.4.2 (pinned to two cores, 1,000 to 1,000,000 strings): 4.6 to 48 times as fast, 12.6 times on average.

Operations that do not accept text

String support does not give every numerical operation a string meaning. NumSharp follows NumPy's dtype-specific refusals:

np.sum(names);    // TypeError: the resolved dtypes are not compatible with add.reduce. Resolved (dtype('<U4'), dtype('<U4'), dtype('<U8'))
np.sqrt(names);   // TypeError: ufunc 'sqrt' not supported for the input types, ...

Those reductions differ for StringDType, which has concatenation and lexical-selection loops. The NumSharp expression engine, np.evaluate, computes on text as NumPy does: S, U and StringDType operands (a C# string is NumPy's str) take every node, anywhere in an expression, and each node runs as NumPy's own function — the six comparisons, the truth value (LogicalAnd / LogicalOr / LogicalXor, Where's condition, Any / All / CountNonzero), IsNaN, Cast to and from text, the np.strings functions as NDExpr.StrLen, NDExpr.IsAlpha … NDExpr.IsNumeric, NDExpr.Find / RFind / Count / Index / RIndex and NDExpr.StartsWith / EndsWith, concatenation (+), StringDType repetition (*), Maximum / Minimum over StringDType, Where with text branches (NumPy's promotion) and the reductions (ArgMax / ArgMin over every class, Sum / Max / Min over StringDType). Its result, text or numbers, feeds the rest of the expression in the same call, and the call's out=, where=, dtype= and order= apply to a text result too. Where NumPy has no loop (the square root of text, bytes plus str), np.evaluate raises NumPy's own error; only NDExpr.Call refuses text, because a delegate's parameters are numbers (StringComputationNotSupportedException):

var t = names.astype(np.dtypes.StringDType());
t == t;                               // [ True  True  True]
np.unique(t);                         // ['bird' 'cat' 'dog']
np.evaluate(NDExpr.Equal(names, "cat") & ((NDExpr)np.array(new[] { 4, 2, 7 }) > 3));   // [ True False False]
np.evaluate(NDExpr.CountNonzero(NDExpr.StrLen(names) > 3));          // 1
np.evaluate(NDExpr.Where(NDExpr.StartsWith(names, "b"), 1.0, 0.5)); // [0.5 0.5 1. ]
np.evaluate((NDExpr)names + names);   // ['catcat' 'dogdog' 'birdbird']
np.evaluate(NDExpr.Where(NDExpr.StrLen(names) > 3, names, "-"));   // ['-' '-' 'bird']
np.evaluate(NDExpr.ArgMax(names));    // 1
np.evaluate(NDExpr.Sqrt(names));      // TypeError: ufunc 'sqrt' not supported for the input types, ...

A comparison of text with numbers is "never equal" in NumPy, and in NumSharp: names == np.array(new[] { 1, 2, 3 }) is [False False False].

numpy.polynomial takes text operands the way NumPy does, up to the first statement that computes with the strings themselves. Its checks and the statements before that point raise or answer as in NumPy: as_series' common-type refusal, polyline's stacking (polyline(np.array("x"), 1) is ['x' '1']), the derivative's and integral's argument checks, the first statement of the *val family and the x + 0.0 of the Vandermonde and fit functions. NumPy then repeats and concatenates the Python strings (polyder(['ab', 'c', 'de']) is ['c' 'dede']) or raises CPython's TypeError from that arithmetic. NumSharp raises NotSupportedException there, because its polynomial package does not model Python's string arithmetic.

NumSharp also reports the problems of a call in NumPy's order, before it answers:

  1. The dtype= request.
  2. A read-only out=, then a non-bool where=.
  3. The axis.
  4. The operation's dtype error.
np.sum(names, 9);   // AxisError: axis 9 is out of bounds for array of dimension 1

An unsized string dtype= request ("U", "S") gets NumPy's answer too. np.sqrt(x, dtype: "U") names no loop. np.add(x, x, dtype: "U") writes the numbers as text and concatenates them.

When the shape decides the answer

Some calls never read a character, because the shape or dtypes decide the result:

  • An empty array. Truth tests, counts, index searches, sorts, np.unique, np.isin, the set routines and the binary ufuncs return NumPy's empty or identity result.
  • Lines of one element. A sort, a partition or an arg-reduction over lines of one element compares nothing.
  • A pair with no comparison loop. Text against numbers is never equal, so np.isin and np.array_equal return False.
var empty = np.zeros(new Shape(0L, 3L), "U2");
np.any(empty);                                  // False
np.argmax(np.array(new[,] { { "b" }, { "a" } }), 1);   // [0 0]
np.isin(names, np.array(new[] { 1.5, 2.0 }));   // [False False False]
np.array_equal(names, np.zeros(new Shape(3L))); // False

np.place and np.putmask follow NumPy's two value forms. A C# array is a Python list: NumSharp converts it straight into the target's dtype. An NDArray must cast to the target's dtype under the 'safe' rule, as in NumPy:

var a = np.arange(5).astype(np.int32);
var mask = np.ones(new Shape(5L), np.@bool);
np.place(a, mask, new[] { 1.9, 2.9 });            // a list: [1 2 1 2 1]
np.place(a, mask, np.array(new[] { 1.9, 2.9 }));  // InvalidCastException (NumPy's TypeError):
                                                  // Cannot cast array data from dtype('float64') to dtype('int32') according to the rule 'safe'

Masked text arrays

np.ma works on text arrays as numpy.ma does. The default fill value is 'N/A', stored at its own width (<U3 or |S3). filled() cuts it to the width of the array:

var m = np.ma.array(np.array(new[] { "ab", "c", "de" }), mask: new[] { true, false, false });
m.fill_value;      // "N/A"
m.filled();        // ['N/' 'c' 'de']
m.filled("zz");    // ['zz' 'c' 'de']
np.ma.count(m);    // 2

The rules for computing are the same as for plain arrays. A masked reduction refuses with NumPy's message. A number used as a fill value is written as its text:

np.ma.sum(m);      // TypeError: the resolved dtypes are not compatible with add.reduce. Resolved (dtype('<U2'), dtype('<U2'), dtype('<U4'))
m.filled(5);       // ['5' 'c' 'de']

For application-specific processing in .NET, NDArray.AsStringArray and tolist() provide managed values. For NumPy's text semantics, use np.strings or np.@char; their Unicode and output-width rules differ from a culture-dependent LINQ transformation.


Variable-width text: StringDType

NumPy 2's StringDType (T, NEP 55) has no fixed text width. Each element is a 16-byte cell: up to 15 UTF-8 bytes sit inside it, longer text lives in an arena owned by the buffer, and a longer rewrite may move the text to the heap. NumSharp follows NumPy's packed layout, missing-data and coercion rules. itemsize is 16 and nbytes counts cells only, not all text memory. tobytes() is not a portable serialization: cells can contain offsets or process-local pointers. Use np.save or the supported text readers and writers to persist values; never reconstruct a T array by copying those cells into another buffer.

StringDType uses the same np.strings operations as fixed-width text, with lengths and search positions measured in Unicode code points. It also supports comparisons, truth values, concatenation, repetition, lexical reductions, sorting, searching, unique values and set routines. Missing values have operation-specific behavior, described below.

Creating StringDType arrays

var t = np.array(new[] { "a", "a much longer text" }, np.dtype("T"));
t.dtype;                                            // StringDType()
t.dtype.itemsize;                                   // 16: the cell, whatever the length of the text
np.zeros(new Shape(2), np.dtypes.StringDType());    // ['', '']
np.ones(new Shape(2), np.dtypes.StringDType());     // ['1', '1']

Missing data: the NA object

np.dtypes.StringDType(na_object: …) names the object that stands for a missing element: null (NumPy's None), double.NaN, a string, or another object. Without an NA object, None is just the text 'None':

var withNA = np.dtypes.StringDType(na_object: null);          // NumPy's StringDType(na_object=None)
var a = np.array(new object[] { "x", null }, withNA);
a.item(1);                                                     // null: the NA object
a.ToString(true);                                              // array(['x', None], dtype=StringDType(na_object=None))
np.array(new object[] { "x", null }, np.dtype("T")).item(1);   // "None"

Writing elements

A value written into a StringDType element is its str(). With coerce: false, only text is accepted:

var t = np.array(new[] { "a", "b" }, np.dtypes.StringDType());
t[0] = 3.25;                       // '3.25'
var strict = np.array(new[] { "a" }, np.dtypes.StringDType(coerce: false));
strict[0] = 5;                     // ValueError: StringDType only allows string data when string coercion is disabled.

Moving and casting

Views share the buffer and its arena. A copy, a gather (np.take, fancy indexing, a boolean mask) and a scatter (np.put, np.putmask, np.place, np.choose) duplicate every string into the destination's own buffer. The casts follow NumPy: text, bool and numbers cast to StringDType (an S text only when its bytes are valid UTF-8, otherwise NumPy's TypeError: Invalid UTF-8 bytes found, cannot convert to StringDType); StringDType casts to a sized U or S (the text is cut to the width) and to numbers (each text is parsed by Python's int(), float() or complex(); a missing element raises):

var t = np.array(new[] { "12", "-3" }, np.dtypes.StringDType());
t.astype(np.int64);                                               // [12, -3]
t.astype("U1");                                                   // ['1', '-']
np.array(new[] { 1.5, 2.0 }).astype(np.dtypes.StringDType());     // ['1.5', '2.0']

A copy moves a short text's cell as one block and reserves the destination's arena once for the longer texts. Measured against NumPy 2.4.2 on the same machine, copies, gathers (np.take, a boolean mask), np.concatenate, np.where, np.repeat, np.array from a list and the casts between U and StringDType run 1.8 to 31 times as fast as NumPy, for 1,000 to 1,000,000 elements.

To an embedded NumPy and back, a StringDType array crosses as an exact copy through NumPy's own C API, with its NA values, na_object and coerce; see Crossing to Python.

Iterating with np.nditer

np.nditer iterates a StringDType operand when you give it refs_ok, as NumPy requires. Buffering and casting work as they do for any other dtype: each buffer has its own string allocator, a buffer fill duplicates every string or runs the cast of the pair (StringDType to U, S, bool or numbers and back; an NA value follows the cast's NA rules), and a written operand gets its strings back when the iterator moves on or closes. it[i] = value stores like a[...] = value: null is the NA object, or 'None' without one.

var t = np.array(new[] { "a", "a long string over fifteen" }, np.dtypes.StringDType());
using (var it = np.nditer(t, new[] { "refs_ok", "buffered" }, op_dtypes: new DType[] { "U5" }, casting: "unsafe"))
    foreach (var x in it)
        Console.WriteLine(x[0].item());                       // a, then a lon (cut to U5)
using (var it = np.nditer(t, new[] { "refs_ok", "buffered" }, new[] { "readwrite" }))
    foreach (var x in it)
        x[0][Slice.Ellipsis] = (string)x[0].item() + "!";      // t is ['a!', 'a long string over fifteen!']

Computing on StringDType text

The six comparisons, the truth values (astype(bool), np.any, np.nonzero, logical_and / logical_or / logical_xor, np.where), np.isnan, add, multiply, maximum, minimum and the reductions np.sum, np.cumsum, np.max, np.min, np.argmax, np.argmin and np.trace compute over StringDType text with NumPy's results, for every memory layout, with out=, where=, axis= and keepdims= where NumPy takes them. A U array or a C# string (NumPy's str) may meet a StringDType array in a comparison and in add; maximum and minimum take two StringDType arrays only, as in NumPy. Missing data follows the NA object:

  • A NaN-like NA (na_object: double.NaN) makes every comparison with it False, != included. Concatenation and repetition propagate it. Lexical selection treats it as greater than every text: maximum / max select NA, while minimum / min select a text when one is available. argmax returns the last NaN-like NA, not the first.
  • A string NA (na_object: "NA") computes as its text.
  • A None-like NA (na_object: null) is equal to another NA and unequal to any text; an ordering or arithmetic with it raises NumPy's ValueError.
var nan = np.dtypes.StringDType(na_object: double.NaN);
var t = np.array(new object[] { "pear", "fig", double.NaN }, nan);
var u = np.array(new object[] { "kiwi", "apple", "plum" }, nan);
t < "grape";                    // [False  True False]
t != u;                         // [ True  True False]: even != is False with the NA
t + "!";                        // ['pear!' 'fig!' nan]
t * 2;                          // ['pearpear' 'figfig' nan]
np.maximum(t, u);               // ['pear' 'fig' nan]
np.minimum(t, u);               // ['kiwi' 'apple' 'plum']
np.sum(t["0:2"]).item();        // "pearfig"
np.cumsum(t);                   // ['pear' 'pearfig' nan]
np.argmax(t);                   // 2: the NA orders last
np.isnan(t);                    // [False False  True]
np.maximum(t, "x");             // TypeError: ufunc 'maximum' did not contain a loop with signature matching types (<class 'numpy.dtypes.StringDType'>, <class 'numpy.dtypes.StrDType'>) -> None

Texts compare as NumPy's NpyString_cmp compares them: memcmp over the shorter length, then the sizes. The first byte that differs decides, a NUL included: two texts are equal only when their bytes are ('a\0c' != 'a\0b', and 'a\0b' < 'a\0c'), a NUL is smaller than any byte it meets ('a\0' < 'ab'), and a text orders before every longer text that starts with it. Fixed-width U text compares every code point too. np.array_equal and np.array_equiv are (a1 == a2).all() with these rules: a NaN-like NA is never equal unless equal_nan: true matches the NA positions (through np.isnan), and a pair that has no comparison loop (S, numbers) is NumPy's all-False array, so only an empty one is equal:

var t2 = np.dtypes.StringDType();
np.array(new object[] { "a\0c" }, t2) == np.array(new object[] { "a\0b" }, t2);   // [False]
np.array_equal(np.array(new object[] { "a\0c", "x" }, t2), np.array(new object[] { "a\0b", "x" }, t2));   // False
np.array_equal(t, t);                                           // False: the NA is not equal to itself
np.array_equal(t, t, equal_nan: true);                          // True

A result takes the operands' common NA object (two different NA objects have no loop) and the first operand's coerce. out= follows NumPy's descriptor resolution: a StringDType out receives the loop's result as it is, and an out of another dtype receives the result of the default StringDType() through NumPy's output cast, so an NA becomes '' in a U or S out:

np.add(t, "!", np.empty(new Shape(3), np.dtype("U8")));   // ['pear!' 'fig!' '']

Two short texts compare inside their 16-byte cells, a text in the arena is read only as far as its first difference, and a new result reserves its arena once at the exact size its texts take. Measured against NumPy 2.4.2 on the same machine (pinned to two cores, 1,000 and 100,000 elements, texts of 6 to 24 bytes: ASCII, non-ASCII, with a long common prefix, with missing elements): all 200 cells run 1.5 to 47 times as fast as NumPy, 3.3 times in the geometric mean. Comparisons run 1.6 to 4.5 times as fast, add / multiply 2.0 to 3.7, maximum / minimum 1.9 to 3.6, truth values and np.isnan 1.7 to 6.3, np.max / np.min 1.5 to 4.4, np.cumsum 1.8 to 5.9, np.argmax 4.9 to 15 and np.sum 2.7 to 47.

String operations with missing values

A missing value is not automatically an empty string. Its behavior depends on the operation and the descriptor's na_object:

  • A string sentinel is processed as that string.
  • A NaN-like sentinel propagates through transformations such as stripping, padding and replacement. Character predicates return False for it. Length and substring-search functions raise because the missing value has no text to measure or search.
  • A None-like sentinel is refused by operations that require text, with NumPy's operation-specific error.
  • Case conversion, formatting, translation and codec functions use the scalar object that an element represents. A non-string missing object can therefore raise a TypeError instead of propagating; for example, upper cannot apply a string method to a NaN or None.

Choose the sentinel for the operations you need, or replace missing values explicitly before transforming text.

Ordering StringDType text

np.sort, np.argsort, ndarray.sort, np.partition, np.argpartition, np.lexsort, np.searchsorted, np.unique and the Array-API unique family, np.isin, the set routines, np.median, np.quantile, np.percentile, their nan forms and numpy.ma's sort / argsort / median / unique order StringDType text with NumPy's results, for every memory layout and axis. Texts compare by their UTF-8 bytes, a text before every longer text that starts with it (NumPy's NpyString_cmp, see Computing on StringDType text), so equal texts are identical texts and every route of np.unique keeps one entry per text. Missing data follows the NA object, as NumPy's sort compares it:

  • A NaN-like NA sorts after every text. The stable kinds preserve the NAs' input order; the default introsort gives NumPy's algorithm-dependent permutation.
  • A string NA reads as its text (the descriptor's default string), and np.unique keeps it and that text as one entry.
  • A None-like NA raises NumPy's ValueError at the first comparison that meets it, so in any line of two or more elements. A line of one element compares nothing.
  • np.unique keeps one entry for all the NAs of a descriptor whose NA object is not NaN-like, whatever equal_nan says; NaN-like NAs are one entry under equal_nan: true (the default) and one each without it.
var nan = np.dtypes.StringDType(na_object: double.NaN);
var t = np.array(new object[] { "pear", double.NaN, "fig", "apple", "fig" }, nan);
np.sort(t);                                                // ['apple' 'fig' 'fig' 'pear' nan]
np.argsort(t, kind: "stable");                             // [3 2 4 0 1]
np.unique(t);                                              // ['apple' 'fig' 'pear' nan]
np.searchsorted(np.sort(t), np.array(new object[] { "grape" }, nan));   // [3]
np.isin(t, np.array(new object[] { "fig" }, nan));         // [False False  True False  True]
var none = np.dtypes.StringDType(na_object: null);
np.sort(np.array(new object[] { "b", null }, none));       // ValueError: Cannot compare null that is not a nan-like value
np.sort(t, descending: true);                              // RuntimeError: unsupported sort kind

NumPy's default kind is an introsort, and NumSharp reproduces its order of equal texts, as for S and U. kind: "heapsort" sorts as the default kind (NumPy clears the heapsort bit before it looks the sort up). descending: true raises NumPy's RuntimeError, because StringDType registers no descending sort; S and U text sorts descending.

The order statistics compute what NumPy computes, and refuse where NumPy refuses:

  • np.median sorts, then takes the mean of the middle elements. Their sum is text, whose division NumPy refuses with a TypeError, so a median exists only when the middle elements are NaN-like NAs (NaN).
  • np.quantile and np.percentile pick an element with the selection methods, as for S and U. An interpolating method reads the two neighbours and subtracts them, which raises Python's TypeError (unsupported operand type(s) for -: 'str' and 'str').
  • The nan forms drop the NaN-like NAs first. An all-NaN slice gives the NA itself.
var s = np.array(new[] { "b", "a", "c" }, np.dtypes.StringDType());
np.quantile(s, 0.5, method: "lower");   // 'b'
np.quantile(s, 0.5);                    // TypeError: unsupported operand type(s) for -: 'str' and 'str'
np.median(s);                           // TypeError: ufunc 'divide' not supported for the input types, ...

np.unique_values (and np.unique with sorted: false and no return_* flag) hashes the texts in NumPy and returns them in its hash table's order; NumSharp returns them sorted, the same values. An axis over two or more dimensions raises NumPy's TypeError, because its structured path cannot hold StringDType fields.

Measured against NumPy 2.4.2 on the same machine (pinned to two cores, fresh inputs every call, 1,000 and 100,000 texts of five kinds: random 8-letter words, 8 repeated words, 24-byte URLs that live in the arena, 6-character non-Latin words, and words with missing elements): all 168 calls measured ran 1.8 to 32 times as fast as NumPy, 7.8 times in the geometric mean. The sorts and partitions ran 2.2 to 32 times as fast, np.lexsort 6.5 to 22, np.searchsorted 3.4 to 5.8, np.unique 2.2 to 13, np.unique_values 1.8 to 4.6, np.isin 9.6 to 24, the set routines 2.9 to 12 and the quantiles 9 to 21. NumPy's sorts call a comparison function for every pair they compare; NumSharp sorts 8-byte prefix keys and replays NumPy's order of equal texts only where it can differ. np.unique hashes the texts when they repeat and sorts them when they are mostly distinct, judged from the repeats among its first texts (an eighth of the array, 256 to 4,096). (np.intersect1d was measured with assume_unique: true: on these inputs NumPy's default path crashes in the sort of the joined unique values, the heapsort-fallback bug listed under "What StringDType refuses".)

Text files: loadtxt, savetxt and tofile

np.loadtxt reads every field of a StringDType column whole, whatever its length. A field equal to a string NA object is a missing element, and a converter's result is stored as a[...] = value stores it. So with na_object: null, a converter that returns null makes the element missing:

var na = np.dtypes.StringDType(na_object: null);
var t = np.loadtxt(new[] { "a b", "- c" }, na, converters: (Func<string, object>)(s => s == "-" ? null : s));
t.item(1, 0);                                      // null: the NA object
np.loadtxt(new[] { "a  b" }, np.dtypes.StringDType(), delimiter: ",").item();   // "a  b": the whole line, spaces kept

np.savetxt formats each element as the Python object NumPy's getitem returns: a str, or the descriptor's NA object for a missing element. %s of a missing element is therefore str(na_object), and a numeric conversion takes the NA object's own type:

var w = new StringWriter();
np.savetxt(w, np.array(new object[] { "x", null }, na), "%s");
w.ToString();                                      // "x\nNone\n"
var nanNA = np.dtypes.StringDType(na_object: double.NaN);
np.savetxt(new StringWriter(), np.array(new object[] { double.NaN }, nanNA), "%d");
                                                   // ValueError: cannot convert float NaN to integer
np.savetxt(new StringWriter(), np.array(new[] { "x" }, na), "%d");
                                                   // TypeError: Mismatch between array dtype ('StringDType(na_object=None)') and format specifier ('%d')

ndarray.tofile(path, sep: ",") writes each element's str(), or format % element. Its binary mode (sep: ""), np.fromfile and np.fromstring refuse a StringDType, as NumPy does:

t.tofile(path, sep: ",");                          // a,b,None,c
np.fromfile(path, np.dtypes.StringDType(), sep: ",");   // ValueError: Cannot read into object array

All three follow NumPy's text encodings (see Reading and writing text files): a file name is read and written as UTF-8 by default, a byte Stream as Latin-1, and encoding: accepts every name Python's codecs.lookup accepts for UTF-8, UTF-16, UTF-32, Latin-1, ASCII and the single-byte code pages.

Measured against NumPy 2.4.2 on the same machine (pinned to two cores, 1,000 to 1,000,000 rows of two StringDType columns): np.loadtxt from a list of lines or a file runs 2.9 to 3.8 times as fast as NumPy, np.savetxt to a writer or a byte stream 5.3 to 8.1 times, and np.save / np.load of the result 2.5 to 7.7 times.

What StringDType refuses

Call NumSharp
view(another dtype), memoryview (.data), np.frombuffer(..., "T") NumPy's errors: a cell is not raw text
view(dtype) / getfield(dtype) with an equal but separate StringDType() instance NumPy's TypeError (Cannot change data-type for array of references., Cannot get/set field of an object array): the cells point into the array's own allocator, so only the array's own dtype object views them
ndarray.byteswap(), in place or not NumPy's TypeError: dtype StringDType() does not implement copyswap: a cell has no byte order (the descriptor's byteorder is '|')
np.ndarray(shape, "T", buffer: ...) over raw bytes or a non-StringDType array, another dtype over a StringDType array's cells, or an offset that splits a cell NotSupportedException. NumPy reads the bytes as cells; a cell holds an arena offset or a heap pointer, so NumSharp refuses. Over another StringDType array the call is a view that shares its cells (offsets and strides in 16-byte cells), and without a buffer the array starts with empty strings
np.fromfile, np.fromstring and tofile's binary mode with StringDType NumPy's errors (Cannot read into object array, OSError for the binary write)
np.unique(..., axis: k) over two or more dimensions NumPy's TypeError: its structured path cannot hold StringDType fields
A np.nanquantile / np.nanmedian row whose result is a None NA NotSupportedException. NumPy builds an object array, a dtype NumSharp does not have

NumPy 2.5.4 has bugs on StringDType that NumSharp does not reproduce; NumSharp gives the correct answer:

  • np.lexsort with a StringDType key that is not contiguous along the axis (a transposed 2-D key, a stepped or reversed view) crashes the process (an access violation).
  • np.argsort(view, axis=None) of a StringDType view that is not C-contiguous (broadcast, reversed, transposed, stepped) misorders texts that live in the arena (a 17-byte text lands before its 16-byte prefix). The same view copied, np.sort, an explicit axis, the partitions and ndarray.sort sort right.
  • The default-kind np.sort / ndarray.sort never returns when its introsort spends its depth budget: the heapsort fallback reads its method context as an array (NumPy 2.4.2 crashed the process there; np.argsort of the same input answers). NumSharp's heapsort answers, in the order np.argpartition gives.
  • A short string copied onto itself (a[...] = a[:1]) becomes ''.
  • np.loadtxt keeps the StringDType instance it is given as its result's descriptor, so a second call with the same instance returns an array NumPy cannot read (MemoryError: Failed to load string in StringDType getitem). NumSharp gives every result its own allocator.
  • np.maximum / np.minimum with a None-like NA into an S out whose texts are not all ASCII raise SystemError: <class 'UnicodeEncodeError'> returned a result with an exception set: the output cast runs after the loop has already failed. NumSharp raises the loop's ValueError: Cannot compare null that is not a nan-like value.

.npy files

np.save and np.load read and write U and S arrays byte for byte, as NumPy writes them. A big-endian >U3 file is byte-swapped when it is read.

Breaking change: a <U1 file now loads as a <U1 string array, not as Char. A Char array is still written as <U1, because NumPy has no 2-byte character dtype.

var stream = new MemoryStream();
np.save(stream, names);
stream.Position = 0;
np.load_npy(stream);    // array(['cat', 'dog', 'bird'], dtype='<U4')

A StringDType array has no fixed-size bytes to write, so NumPy writes a pickle: the header names |O, and the data is pickle.dump(array, protocol=4). np.save writes the same bytes as numpy.save, and np.load(allow_pickle: true) reads them, NA values, na_object and coerce included. As in NumPy, loading without allow_pickle refuses. .npz archives hold StringDType members the same way.

var words = np.array(new object[] { "cat", null }, np.dtypes.StringDType(na_object: null));
var stream = new MemoryStream();
np.save(stream, words);                     // the bytes numpy.save writes
stream.Position = 0;
np.load_npy(stream, allow_pickle: true);    // array(['cat', None], dtype=StringDType(na_object=None))
stream.Position = 0;
np.load_npy(stream);                        // FormatException: Object arrays cannot be loaded when allow_pickle=False

NumSharp reads only the pickle NumPy writes for a StringDType array. It has no general unpickler: a pickle of anything else (an object array, a Python dict, a call to a Python function) is refused, and nothing in the file runs. np.save(..., allow_pickle: false) raises NumPy's ValueError: Object arrays cannot be saved when allow_pickle=False, and mmap_mode raises ValueError: Array can't be memory-mapped: Python objects in dtype.


Crossing to Python

With NumSharp.Interop.pythonnet, text crosses to an embedded NumPy the way numbers do, and a StringDType array as an exact copy:

Array NumSharp → NumPy NumPy → NumSharp
S<n>, U<n> ToNumpy() shares the bytes (\|S<n>, <U<n>) AsNDArray() shares numpy's bytes
StringDType ToNumpyCopy() copies, NA values and descriptor included ToNDArray() copies, NA values and descriptor included
NumPy object array of str - ToNDArray() copies as StringDType()
var words = np.array(new object[] { "cat", null }, np.dtypes.StringDType(na_object: null));
using (Py.GIL())
{
    using PyObject u = names.ToNumpy();       // array(['cat', 'dog', 'bird'], dtype='<U4'), the same memory
    using PyObject t = words.ToNumpyCopy();   // array(['cat', None], dtype=StringDType(na_object=None))
}

A StringDType array has no shared form: NumPy keeps each element's text in an allocator its descriptor owns, so the bridge copies it element by element through NumPy's own NpyString C API. The missing-data object crosses when C# can hold it (None, bool, int, float, complex, str, bytes, np.float16); any other object, such as pd.NA, is refused.

Breaking change: NumPy's <U1 (and a 4-byte wchar_t buffer) now imports as a U1 view, not as a Char copy. A U1 element is a 4-byte code point and holds an astral character that a char cannot.


The Char dtype

NPTypeCode.Char wraps .NET's System.Char, a 2-byte UTF-16 code unit. It stores characters as numbers, not text:

var letters = np.array(new[] { 'a', 'b', 'c' });
letters.typecode;        // NPTypeCode.Char
InfoOf<char>.Size;       // 2  (actual memory footprint)

Char is not NumPy's <U1 or S1. NumPy's <U1 is a 4-byte code point, and S1 is one byte. NumSharp's Char is a 2-byte UTF-16 unit that behaves like a 2-byte integer in arithmetic. For text, use a U array. The descriptor reports itemsize == 2 and kind == 'u'. See Data types → Char.


Byte streams → byte[] + np.frombuffer

For raw bytes, use byte[] and reinterpret it with np.frombuffer when you want to view the bytes as numbers:

byte[] raw = File.ReadAllBytes("data.bin");
var asBytes  = np.frombuffer(raw, np.uint8);     // NDArray<byte>, zero-copy view
var asFloats = np.frombuffer(raw, np.float32);   // reinterpret the same bytes as float32

This is NumPy's np.frombuffer on a bytestring, and the general interop bridge. See Any library via np.frombuffer.


Porting cheat-sheet

NumPy NumSharp
np.array(["hello", "world"]) (→ <U5) np.array(new[] { "hello", "world" }) (→ <U5)
np.array([b"hi"], dtype="S2") np.array(new object[] { Encoding.ASCII.GetBytes("hi") }, np.dtype("S2"))
arr.astype("U10") arr.astype("U10")
np.arange(3).astype(str) (→ <U21) np.arange(3).astype("U") (→ <U21)
np.array(["1", "2"]).astype(float) np.array(new[] { "1", "2" }).astype(np.float64)
arr[0] (a numpy.str_) arr.item(0) (a C# string)
arr.tolist() arr.tolist() or NDArray.AsStringArray(arr)
a + b / np.char.add(a, b) / np.strings.add(a, b) a + b / np.@char.add(a, b) / np.strings.add(a, b)
np.char.multiply(a, 3) / np.strings.multiply(a, 3) np.@char.multiply(a, 3) / np.strings.multiply(a, 3)
np.char.equal(a, b) (strips trailing whitespace) np.@char.equal(a, b)
np.char.compare_chararrays(a, b, '<', True) np.@char.compare_chararrays(a, b, "<", true)
c = np.char.array(["a ", "b"]); c[0] (the stripped str) var c = np.@char.array(new[] { "a ", "b" }); c[0].item()
a.view(np.char.chararray) a.view(typeof(NDCharArray))
c == "ab", u == c (a chararray strips on either side) the same, with c of type NDCharArray (in an NDArray variable, == does not strip)
c.argmax(out=o) c.argmax(null, o) (without out, ValueError as in NumPy)
np.char.upper(arr) / np.strings.upper(arr) np.@char.upper(arr) / np.strings.upper(arr)
np.strings.find(arr, "ab") / np.strings.strip(arr) the same calls
np.strings.encode(arr, "utf-8") np.strings.encode(arr, "utf-8")
np.strings.partition(arr, "-") three result arrays; np.@char.partition(arr, "-") stacks them on the last axis
np.array(["a", "bb"], dtype=StringDType()) np.array(new[] { "a", "bb" }, np.dtypes.StringDType())
StringDType(na_object=None) np.dtypes.StringDType(na_object: null)
np.array(['a','b','c']) (single characters, <U1) np.array(new[] { "a", "b", "c" }) (<U1); np.array(new[] { 'a', 'b', 'c' }) is Char

Troubleshooting

"Why does np.evaluate raise a TypeError on my string array?"

np.evaluate computes on text exactly where NumPy does, so it raises exactly where NumPy raises, with NumPy's text: a node NumPy has no loop for (NDExpr.Sqrt(names), NDExpr.Subtract(names, names), NDExpr.LogicalNot(names)), operands NumPy will not mix (bytes with str, text with a number in NDExpr.Add, a Python number beside fixed-width text in NDExpr.Where), or a reduction NumPy refuses for that class (NDExpr.Sum of U text, a flat NDExpr.Sum of StringDType over several axes). The error comes when the call runs, in the order NumPy's statements would raise it. Only an NDExpr.Call delegate refuses text in its own words (StringComputationNotSupportedException): its parameters are numbers, so measure or compare the text in the expression (NDExpr.StrLen, NDExpr.Equal, ...) and pass that number in.

"ValueError: could not convert string to float"

A cast from text to a number parses every element, as NumPy does. Clean the text first, or keep it as text.

"My NDArray x = "1.5" is text, not a number"

A C# string is NumPy's str: a 0-d U array. Write the number itself (NDArray x = 1.5), or parse the text with astype(np.float64).

"My byte[][] became a uint8 matrix"

A jagged byte[][] is numeric data in C#. Pass an object[] of byte[] values to get an S array.

"My value was cut"

An S / U array has a fixed width. Writing a longer value cuts it, as NumPy does. Use a wider descriptor (arr.astype("U10")) or convert to np.dtypes.StringDType() when future writes must grow.

"UnicodeEncodeError when I cast to S"

A U to S cast encodes ASCII only, as NumPy does. Keep non-ASCII text in a U array.

"GetString threw InvalidOperationException"

GetString reads Char vectors. Read a string element with item(i) or NDArray.AsStringArray.


API reference

Item Behavior
np.dtype("U5") / np.dtype("S5") The fixed-width descriptors: itemsize 20 / 5, kind 'U' / 'S'
np.array(string[]) A U array as wide as the longest value
np.array(object[] of byte[]) An S array as wide as the longest value
NDArray.Scalar(string) / NDArray.Scalar(byte[]) A 0-d U / S array
arr.item(i) A C# string (U) or byte[] (S), trailing NULs removed
NDArray.AsStringArray(arr) Every element of a U array as a string[], in C order
arr[i] = "text" Writes one element, cut to the width
arr.astype("S3") / arr.astype("U8") Casts between string dtypes, as NumPy does
arr.astype("U") of numbers Each number as its scalar's str(), at NumPy's width for the number type (21 for int64, 32 for float64)
arr.astype(np.float64) of text Each text parsed by float() (int() / complex() for those targets), with NumPy's errors
arr[i] = 5 / arr[i] = "12" A number stored as text / a C# string parsed into a numeric array
Comparisons (==, <, np.equal, ...) and truth values (astype(bool), np.any, np.nonzero, ...) NumPy's results
np.add(a, b) / a + b / np.strings.add(a, b) over two text arrays of one class The concatenation, as wide as the two widths together
np.strings.multiply(a, i) / np.multiply(a, i, out) The repetition, as wide as the longest one / cut to the width of out
np.strings.str_len, find, rfind, index, rindex, count, startswith, endswith Elementwise lengths, substring positions, counts and prefix/suffix tests
np.strings.strip, lstrip, rstrip, replace, expandtabs, center, ljust, rjust, zfill, slice Elementwise text transformations with each function's output-width rule
np.strings.partition / rpartition Three arrays containing the before / separator / after parts
np.strings.isalpha, isalnum, isdigit, isdecimal, isnumeric, isspace, islower, isupper, istitle Boolean character-property tests; isdecimal / isnumeric require Unicode text
np.strings.upper, lower, swapcase, capitalize, title, translate Unicode/ASCII transformations; fixed-width results keep their input dtype
np.strings.mod, encode, decode Python-style formatting and codec conversion, sized to the resulting text
np.@char string functions and NDCharArray string methods The legacy interface to the same text operations, with chararray return rules and stacked partition results
np.sort / np.argsort / arr.sort() / np.partition / np.argpartition NumPy's results, NumPy's order of equal strings for every kind, NumPy's argument errors
np.@char.equal … np.@char.greater_equal, np.@char.compare_chararrays Comparisons after the trailing NULs and ASCII whitespace are removed; null (NumPy's NotImplemented) for S against U
np.@char.add / np.@char.multiply np.add / np.strings.multiply; a count that is not an integer is ValueError("Can only multiply by integers")
np.@char.chararray / np.@char.array / np.@char.asarray NumPy's chararray (NDCharArray): element access and iteration remove trailing whitespace, views and copies stay chararrays, text data only; the comparison operators strip (also with a plain array on the left), + and * return chararrays, and the np.* functions keep, wrap or drop the class as NumPy's do
np.@char.split / rsplit / splitlines Python's str / bytes methods per element; the result is an NDListArray, NumPy's object array of lists
np.@char.join The characters of each string joined by its separator; the result has the class of seq; over an NDListArray, NumPy 2.5.4's TypeError (over an empty one, NotSupportedException: NumPy's empty object array)
np.argmax / np.argmin / np.searchsorted / np.digitize NumPy's results: the first extreme element, NumPy's exact search over unsorted haystacks
np.unique / np.unique_values / np.unique_counts / np.unique_inverse / np.unique_all NumPy's results for every flag and axis; unique_values (and sorted: false) sorted where NumPy keeps its hash order
np.isin / np.intersect1d / np.union1d / np.setdiff1d / np.setxor1d / np.lexsort / np.sort_complex NumPy's results: numbers beside text written as text, bytes decoded as ASCII where NumPy joins the operands
np.histogram / np.histogram_bin_edges NumPy's results with text or number edges (compared as text), weights and density
np.quantile / np.percentile NumPy's results for the methods that pick an element; the interpolating methods raise NumPy's TypeError
Operations without a text loop (np.sqrt; np.sum of S / U) NumPy's exception and message
np.dtypes.StringDType(...) / np.dtype("T") NumPy's variable-width StringDType: storage, element access, copies, gathers, scatters, casts and printing as NumPy's; comparisons, truth values, add / multiply / maximum / minimum, the reductions, the sorts, searches, unique values, set routines and order statistics compute with NumPy's results and its ordering of missing values
np.dtype("V7") / "O" NotSupportedException
NPTypeCode.Char A 2-byte UTF-16 code unit dtype, written to .npy as <U1
np.frombuffer(byte[], dtype) Reinterprets raw bytes as an array, string dtypes included