Class StickyAction<TObs, TAct>
With probability repeat_action_probability, replaces the agent's action by the previous one for a run of
repeat_action_duration steps, as Gymnasium's StickyAction does (Machado et al., 2018, ยง5.2): the
stochasticity that keeps an agent from memorising a deterministic env's action sequence.
public class StickyAction<TObs, TAct> : ActionWrapper<TObs, TAct, TAct>, IDynamicAttributes, IWrapper, IEnv, IDisposable, IRecordConstructorArgs
Type Parameters
TObsThe observation type, passed through.
TActThe action type, unchanged.
- Inheritance
-
Env<TObs, TAct>Wrapper<TObs, TAct, TObs, TAct>ActionWrapper<TObs, TAct, TAct>StickyAction<TObs, TAct>
- Implements
- Inherited Members
- Extension Methods
Remarks
The draws come from the env's generator (self.np_random), so they shift the env's own stream: a seeded
reset makes them reproducible, and an env that draws in its steps sees other numbers than without the wrapper.
A run starts only when there is a previous action (never at an episode's first step); its length is drawn from
[low, high] when it starts.
Python keeps the previous action object itself. Here the caller owns its actions and may dispose one after the step, so the wrapper keeps a deep copy of its own (LastAction) and hands the env a fresh copy of it at every repeated step, which the env may keep or release; the agent's own actions pass through as they are. A caller that mutates an action array after the step therefore sees its old values repeated, where Python would repeat the mutated array.
repeat_action_duration is a Python int (any .NET integer) or a tuple of two (object?[] or a
C# tuple); a list (any other IList) is refused as Gymnasium refuses it. Bounds
that are floats are accepted as Gymnasium accepts them, and the draw truncates them as NumPy's integers does.
Reference: gymnasium/wrappers/stateful_action.py.
Constructors
StickyAction(Env<TObs, TAct>, StickyActionOptions)
Builds the instance from its options record: the properties set, the constructor's defaults for the rest.
public StickyAction(Env<TObs, TAct> env, StickyActionOptions options)
Parameters
envEnv<TObs, TAct>The env to wrap.
optionsStickyActionOptionsThe options.
Exceptions
- ArgumentNullException
Thrown when
optionsis null.- TypeError
Thrown when a required argument is unset, with Python's missing-argument message.
StickyAction(Env<TObs, TAct>, double, object?)
Wraps env.
public StickyAction(Env<TObs, TAct> env, double repeatActionProbability, object? repeatActionDuration = null)
Parameters
envEnv<TObs, TAct>The env.
repeatActionProbabilitydoubleThe probability that a run of repeats starts at a step; in [0, 1).
repeatActionDurationobjectThe length of a run: an integer (null is Gymnasium's default, 1), or a tuple
(low, high)to draw each run's length from.
Exceptions
- InvalidProbability
Thrown for a probability outside [0, 1) or NaN:
.repeat_action_probabilityshould be in the interval [0,1). Received 1.0- ValueError
Thrown for a duration that is neither an integer nor a tuple, a tuple whose length is not 2, or a bound below 1, with Gymnasium's messages.
- InvalidBound
Thrown when the low bound exceeds the high one:
.repeat_action_durationis not a valid bound. Received (4, 2)- TypeError
Thrown when a bound is not a number (Python's comparison refuses it), or
envis null.
Properties
IsStickyActions
Whether a run of repeats is in progress (is_sticky_actions).
public bool IsStickyActions { get; }
Property Value
LastAction
A copy of the action the env got at the last step (last_action), owned by the wrapper (do not dispose it);
the default before the episode's first step.
public TAct? LastAction { get; }
Property Value
- TAct
NumRepeats
The length of the current run, drawn when it starts; 0 outside a run (num_repeats).
public long NumRepeats { get; }
Property Value
RepeatActionDurationRange
The bounds a run's length is drawn from, a tuple of two (repeat_action_duration_range).
public object?[] RepeatActionDurationRange { get; }
Property Value
- object[]
RepeatActionProbability
The probability that a run of repeats starts (repeat_action_probability).
public double RepeatActionProbability { get; set; }
Property Value
RepeatsTaken
The repeats the current run has taken (repeats_taken).
public long RepeatsTaken { get; }
Property Value
SavedKwargs
The recorded constructor arguments, the duration as the caller gave it (Python's _saved_kwargs).
[PyName("_saved_kwargs")]
public Info SavedKwargs { get; }
Property Value
Methods
Action(TAct)
The action the env gets: the agent's, or a copy of the previous one while a run of repeats lasts.
public override TAct Action(TAct action)
Parameters
actionTActThe agent's action; it stays the caller's.
Returns
- TAct
actionitself, or a new copy of the previous action that the env may keep.
Dispose(bool)
Releases the wrapper's copy of the last action, then disposes the wrapper as every wrapper does.
protected override void Dispose(bool disposing)
Parameters
Reset(BigInteger?, IReadOnlyDictionary<string, object?>?)
Forgets the last action and any run, then resets the env.
public override ResetResult<TObs> Reset(BigInteger? seed = null, IReadOnlyDictionary<string, object?>? options = null)
Parameters
seedBigInteger?The seed, passed on.
optionsIReadOnlyDictionary<string, object>The options, passed on.
Returns
- ResetResult<TObs>
The env's reset.
Exceptions
- Error
Thrown for a negative seed.