Table of Contents

Class StickyAction<TObs, TAct>

Namespace
Gym.Wrappers
Assembly
Gym.dll

With probability repeat_action_probability, replaces the agent's action by the previous one for a run of repeat_action_duration steps, as Gymnasium's StickyAction does (Machado et al., 2018, ยง5.2): the stochasticity that keeps an agent from memorising a deterministic env's action sequence.

public class StickyAction<TObs, TAct> : ActionWrapper<TObs, TAct, TAct>, IDynamicAttributes, IWrapper, IEnv, IDisposable, IRecordConstructorArgs

Type Parameters

TObs

The observation type, passed through.

TAct

The action type, unchanged.

Inheritance
Env<TObs, TAct>
Wrapper<TObs, TAct, TObs, TAct>
ActionWrapper<TObs, TAct, TAct>
StickyAction<TObs, TAct>
Implements
Inherited Members
Extension Methods

Remarks

The draws come from the env's generator (self.np_random), so they shift the env's own stream: a seeded reset makes them reproducible, and an env that draws in its steps sees other numbers than without the wrapper. A run starts only when there is a previous action (never at an episode's first step); its length is drawn from [low, high] when it starts.

Python keeps the previous action object itself. Here the caller owns its actions and may dispose one after the step, so the wrapper keeps a deep copy of its own (LastAction) and hands the env a fresh copy of it at every repeated step, which the env may keep or release; the agent's own actions pass through as they are. A caller that mutates an action array after the step therefore sees its old values repeated, where Python would repeat the mutated array.

repeat_action_duration is a Python int (any .NET integer) or a tuple of two (object?[] or a C# tuple); a list (any other IList) is refused as Gymnasium refuses it. Bounds that are floats are accepted as Gymnasium accepts them, and the draw truncates them as NumPy's integers does.

Reference: gymnasium/wrappers/stateful_action.py.

Constructors

StickyAction(Env<TObs, TAct>, StickyActionOptions)

Builds the instance from its options record: the properties set, the constructor's defaults for the rest.

public StickyAction(Env<TObs, TAct> env, StickyActionOptions options)

Parameters

env Env<TObs, TAct>

The env to wrap.

options StickyActionOptions

The options.

Exceptions

ArgumentNullException

Thrown when options is null.

TypeError

Thrown when a required argument is unset, with Python's missing-argument message.

StickyAction(Env<TObs, TAct>, double, object?)

Wraps env.

public StickyAction(Env<TObs, TAct> env, double repeatActionProbability, object? repeatActionDuration = null)

Parameters

env Env<TObs, TAct>

The env.

repeatActionProbability double

The probability that a run of repeats starts at a step; in [0, 1).

repeatActionDuration object

The length of a run: an integer (null is Gymnasium's default, 1), or a tuple (low, high) to draw each run's length from.

Exceptions

InvalidProbability

Thrown for a probability outside [0, 1) or NaN: repeat_action_probability should be in the interval [0,1). Received 1.0.

ValueError

Thrown for a duration that is neither an integer nor a tuple, a tuple whose length is not 2, or a bound below 1, with Gymnasium's messages.

InvalidBound

Thrown when the low bound exceeds the high one: repeat_action_duration is not a valid bound. Received (4, 2).

TypeError

Thrown when a bound is not a number (Python's comparison refuses it), or env is null.

Properties

IsStickyActions

Whether a run of repeats is in progress (is_sticky_actions).

public bool IsStickyActions { get; }

Property Value

bool

LastAction

A copy of the action the env got at the last step (last_action), owned by the wrapper (do not dispose it); the default before the episode's first step.

public TAct? LastAction { get; }

Property Value

TAct

NumRepeats

The length of the current run, drawn when it starts; 0 outside a run (num_repeats).

public long NumRepeats { get; }

Property Value

long

RepeatActionDurationRange

The bounds a run's length is drawn from, a tuple of two (repeat_action_duration_range).

public object?[] RepeatActionDurationRange { get; }

Property Value

object[]

RepeatActionProbability

The probability that a run of repeats starts (repeat_action_probability).

public double RepeatActionProbability { get; set; }

Property Value

double

RepeatsTaken

The repeats the current run has taken (repeats_taken).

public long RepeatsTaken { get; }

Property Value

long

SavedKwargs

The recorded constructor arguments, the duration as the caller gave it (Python's _saved_kwargs).

[PyName("_saved_kwargs")]
public Info SavedKwargs { get; }

Property Value

Info

Methods

Action(TAct)

The action the env gets: the agent's, or a copy of the previous one while a run of repeats lasts.

public override TAct Action(TAct action)

Parameters

action TAct

The agent's action; it stays the caller's.

Returns

TAct

action itself, or a new copy of the previous action that the env may keep.

Dispose(bool)

Releases the wrapper's copy of the last action, then disposes the wrapper as every wrapper does.

protected override void Dispose(bool disposing)

Parameters

disposing bool

true from Dispose().

Reset(BigInteger?, IReadOnlyDictionary<string, object?>?)

Forgets the last action and any run, then resets the env.

public override ResetResult<TObs> Reset(BigInteger? seed = null, IReadOnlyDictionary<string, object?>? options = null)

Parameters

seed BigInteger?

The seed, passed on.

options IReadOnlyDictionary<string, object>

The options, passed on.

Returns

ResetResult<TObs>

The env's reset.

Exceptions

Error

Thrown for a negative seed.