Class RunningMeanStd
A running mean and variance (RunningMeanStd), updated batch by batch with Chan et al.'s parallel algorithm:
what NormalizeObservation<TObs, TAct> and NormalizeReward<TObs, TAct> normalize by.
Byte for byte Gymnasium's, NumPy 2's dtype rules included: the statistics keep their dtype (float32 for an
observation normalizer) until a batch of a wider dtype promotes them, and the count stays a float64.
public sealed class RunningMeanStd : IDisposable
- Inheritance
-
RunningMeanStd
- Implements
- Inherited Members
- Extension Methods
Remarks
The arrays are this object's: an update replaces them with new ones and disposes the old, so a reference taken from Mean or Var is valid only until the next update. Not thread-safe.
Reference: gymnasium/wrappers/utils.py.
Constructors
RunningMeanStd(double, Shape?, DType?)
Starts with mean 0, variance 1 and a count of epsilon, which keeps the first update's division away from zero.
public RunningMeanStd(double epsilon = 0.0001, Shape? shape = null, DType? dtype = null)
Parameters
epsilondoubleThe initial count, 1e-4 by default.
shapeShape?The statistics' shape,
()by default (a scalar, as for rewards).dtypeDTypeThe statistics' dtype, float64 by default.
Properties
Count
The number of samples seen, plus the initial epsilon (count), a Python float.
public double Count { get; set; }
Property Value
Mean
The running mean (mean); owned by this object and replaced by every update.
public NDArray Mean { get; set; }
Property Value
Var
The running variance (var); owned by this object and replaced by every update.
public NDArray Var { get; set; }
Property Value
Methods
Dispose()
Releases the statistics' arrays.
public void Dispose()
Update(NDArray)
Folds a batch in (update(x)): its mean and variance along the first axis, and its length as the count.
public void Update(NDArray x)
Parameters
xNDArrayThe batch,
(k, *shape); left to the caller.
Exceptions
- ArgumentNullException
Thrown when
xis null.
UpdateFromMoments(NDArray, NDArray, long)
Folds a batch's moments in (update_from_moments).
public void UpdateFromMoments(NDArray batchMean, NDArray batchVar, long batchCount)