Table of Contents

Class AcrobotEnv

Namespace
Gym.Environments.Envs.Classic
Assembly
Gym.Environments.dll

Acrobot-v1: two links joined in a chain, hanging from a fixed joint, with a torque of -1, 0 or +1 (actions 0, 1, 2) on the joint between them; the goal is to swing the free end above a line one link-length over the base. -1 per step until then, 0 on the terminating step. Byte for byte what Gymnasium's Acrobot computes, errors included.

public class AcrobotEnv : Env<NDArray, NDArray>, IEnv, IDisposable, IDynamicAttributes
Inheritance
AcrobotEnv
Implements
Inherited Members
Extension Methods

Remarks

The dynamics are Gymnasium's: the "book" equations (BookOrNips) integrated over Dt by one fourth-order Runge-Kutta step, in float64, every product and sum in the source's order and every x ** 2 the C runtime's pow, as NumPy computes them. The reset state is float32 (uniform(...).astype(np.float32)), so the first observation's cosines and sines are NumPy's own float32 ones (through NumSharp, bit for bit) and the first step starts from the float32-rounded values; after a step the state is float64.

The action is not checked against the action space, as in Gymnasium: it indexes AvailTorque the way a Python list is indexed, so -1, -2 and -3 are valid (the torques of 2, 1 and 0), and other values raise Python's IndexError or TypeError. The physical constants are Gymnasium's class attributes, settable per env here (Python's are shared by the class); RLPy's unused action_arrow, domain_fig and actions_num are not carried.

Rendering is Gymnasium's (the line, the links and joints, pixel for pixel). Human mode renders into the window from viewer_factory or Gymnasium.HumanViewerFactory (DR-20) at 15 frames per second. A state angle that is infinite never wraps, in Gymnasium's wrap as here, so such a step does not return.

Not thread-safe. Reference: gymnasium/envs/classic_control/acrobot.py.

Its info is always {}, as Gymnasium's, so it has no typed info record (DR-95).

Constructors

AcrobotEnv(AcrobotEnvOptions)

Builds the instance from its options record: the properties set, the constructor's defaults for the rest.

public AcrobotEnv(AcrobotEnvOptions options)

Parameters

options AcrobotEnvOptions

The options.

Exceptions

ArgumentNullException

Thrown when options is null.

AcrobotEnv(RenderMode, EnvViewerFactory?)

Builds the env with the render mode given as the RenderMode enum (DR-84, DR-93), as AcrobotEnv(string?, EnvViewerFactory?) builds it from the mode's string: new AcrobotEnv(RenderMode.RgbArray) builds what new AcrobotEnv(renderMode: "rgb_array") builds, the member reaching the env as Gymnasium's string (Name(RenderMode)), which its RenderMode property then holds.

public AcrobotEnv(RenderMode renderMode, EnvViewerFactory? viewerFactory = null)

Parameters

renderMode RenderMode

The render mode, which the env must list in its metadata["render_modes"] as for the string; it behaves exactly as its Gymnasium string does in the other constructor.

viewerFactory EnvViewerFactory

Gym.NET's viewer_factory (DR-20) for human mode; null uses Gymnasium.HumanViewerFactory.

Remarks

The mode is required and comes first, so new AcrobotEnv() keeps meaning the other constructor, without a mode; leave the mode out there for no rendering. Every other parameter means what it means there.

Exceptions

ArgumentOutOfRangeException

Thrown for a value RenderMode does not declare (a cast integer such as (RenderMode)99).

AcrobotEnv(string?, EnvViewerFactory?)

Creates the env, stateless until the first reset.

public AcrobotEnv(string? renderMode = null, EnvViewerFactory? viewerFactory = null)

Parameters

renderMode string

Gymnasium's render_mode: null, "human" or "rgb_array"; not validated, as in Gymnasium (make checks it against the metadata).

viewerFactory EnvViewerFactory

Gym.NET's viewer_factory (DR-20) for human mode; null uses Gymnasium.HumanViewerFactory.

Properties

AvailTorque

Gymnasium's AVAIL_TORQUE, [-1.0, 0.0, +1]: the torque of each action, indexed as a Python list (a negative action counts from the end). Python's last entry is the int 1, which joins the float64 state as 1.0, so a double holds it exactly.

[PyName("AVAIL_TORQUE")]
public double[] AvailTorque { get; set; }

Property Value

double[]

BookOrNips

Gymnasium's book_or_nips, "book": which equation gives the second joint's acceleration. "nips" uses the NIPS paper's, which leaves out a velocity term; any other string the book's (and the Java implementation's).

public string BookOrNips { get; set; }

Property Value

string

ClassMetadata

Gymnasium's class attribute metadata: {"render_modes": ["human", "rgb_array"], "render_fps": 15}. Every env starts with a copy as its instance metadata.

public static Metadata ClassMetadata { get; }

Property Value

Metadata

Remarks

A new dict on every read, so editing one env's metadata never reaches another (Python's is one shared dict).

Clock

Gymnasium's clock: paces human rendering, null until the first render.

public Clock? Clock { get; }

Property Value

Clock

Dt

Gymnasium's dt, 0.2 s: the integration time step, one Runge-Kutta step per env step.

public double Dt { get; set; }

Property Value

double

IsOpen

Gymnasium's isopen: true until Close() shuts a render window down.

[PyName("isopen")]
public bool IsOpen { get; set; }

Property Value

bool

LinkComPos1

Gymnasium's LINK_COM_POS_1, 0.5 m: the first link's center of mass.

[PyName("LINK_COM_POS_1")]
public double LinkComPos1 { get; set; }

Property Value

double

LinkComPos2

Gymnasium's LINK_COM_POS_2, 0.5 m: the second link's center of mass.

[PyName("LINK_COM_POS_2")]
public double LinkComPos2 { get; set; }

Property Value

double

LinkLength1

Gymnasium's LINK_LENGTH_1, 1.0 m. Also sizes the frame (the world spans both links plus 0.2).

[PyName("LINK_LENGTH_1")]
public double LinkLength1 { get; set; }

Property Value

double

LinkLength2

Gymnasium's LINK_LENGTH_2, 1.0 m. Used only by rendering, as in Gymnasium's dynamics.

[PyName("LINK_LENGTH_2")]
public double LinkLength2 { get; set; }

Property Value

double

LinkMass1

Gymnasium's LINK_MASS_1, 1.0 kg.

[PyName("LINK_MASS_1")]
public double LinkMass1 { get; set; }

Property Value

double

LinkMass2

Gymnasium's LINK_MASS_2, 1.0 kg.

[PyName("LINK_MASS_2")]
public double LinkMass2 { get; set; }

Property Value

double

LinkMoi

Gymnasium's LINK_MOI, 1.0: both links' moment of inertia.

[PyName("LINK_MOI")]
public double LinkMoi { get; set; }

Property Value

double

MaxVel1

Gymnasium's MAX_VEL_1, 4π: the first joint's angular velocity bound, applied every step.

[PyName("MAX_VEL_1")]
public double MaxVel1 { get; set; }

Property Value

double

MaxVel2

Gymnasium's MAX_VEL_2, 9π: the second joint's angular velocity bound.

[PyName("MAX_VEL_2")]
public double MaxVel2 { get; set; }

Property Value

double

Screen

Gymnasium's screen: the surface every frame is blitted onto, null until the first render. Close() keeps it, as Python's close does.

public Surface? Screen { get; }

Property Value

Surface

ScreenDim

Gymnasium's SCREEN_DIM, 500: the frame's width and height in pixels.

[PyName("SCREEN_DIM")]
public int ScreenDim { get; set; }

Property Value

int

State

Gymnasium's state, [θ1, θ2, θ̇1, θ̇2]: a new array of the values on every read, null before the first reset. Assigning injects a state, as env.state = np.array([...]) does: float64 values of any length, checked by the next step with Python's errors; null is Python's None.

public double[]? State { get; set; }

Property Value

double[]

Remarks

Python's array is float32 after a reset and float64 after a step; the values read here are exactly its values, a float32 one widened.

TorqueNoiseMax

Gymnasium's torque_noise_max, 0.0: when positive, every step adds uniform(-max, max) to the torque, one draw from the env's generator.

public double TorqueNoiseMax { get; set; }

Property Value

double

Methods

Bound(double, double, double)

Gymnasium's module function bound with scalar limits: min(max(x, m), M) with Python's built-ins, which keep x when a comparison is false (so NaN stays NaN).

public static double Bound(double x, double m, double M)

Parameters

x double

The value.

m double

The lower limit.

M double

The upper limit.

Returns

double

The bounded value.

Remarks

Close()

Once something was rendered, shuts the human-mode window down and marks the env closed (IsOpen). Like Gymnasium's, it keeps Screen, so a later render reuses it.

public override void Close()

Remarks

ConvertAction(object?)

The untyped step's action conversion, with the source's order kept: its state assert first, then the list index on the action as given, so the errors name the Python counterpart of the value given (not float for a double, not numpy.float32 for a float).

protected override NDArray ConvertAction(object? action)

Parameters

action object

The action as given to the untyped step.

Returns

NDArray

The action as a 0-d integer array: the one given, or a new one the step releases.

Exceptions

AssertionError

Thrown before the first reset.

TypeError

Thrown for a value Python cannot index a list with.

IndexError

Thrown for an index outside the list, or an integer beyond 64 bits.

Render()

Draws Gymnasium's frame with the same primitives in the same order: the target line, then each link as a rotated rectangle with a circle at its joint, in a y-up world flipped at the end.

public override RenderResult Render()

Returns

RenderResult

rgb_array: the (500, 500, 3) uint8 frame, a new array the caller owns. human, any other mode, and every mode before the first reset: nothing.

Remarks

After a reset the state is float32, so the first link's end and both angles are computed in float32 (NumPy's float32 cosine and sine), as in Python; after a step, in float64. Reference: gymnasium/envs/classic_control/acrobot.py.

Exceptions

AssertionError

Thrown without a render mode when the env has no spec (assert self.spec is not None).

IndexError

Thrown for an assigned state of fewer than two values, with NumPy's message.

KeyError

Thrown in human mode when metadata["render_fps"] was removed.

ValueError

Thrown for a NaN coordinate (a NaN state), with pygame's or Python's message.

OverflowError

Thrown, as pygame's OverflowError, for a coordinate pygame cannot draw.

DependencyNotInstalled

Thrown on the first human render when no viewer factory is set (DR-20).

Reset(BigInteger?, IReadOnlyDictionary<string, object?>?)

Starts an episode: reseeds the generator when a seed is given, then draws the four state variables from uniform(low, high, size=(4,)) (default ±0.1, or the "low"/"high" options) and rounds them to float32. Renders in human mode.

public override ResetResult<NDArray> Reset(BigInteger? seed = null, IReadOnlyDictionary<string, object?>? options = null)

Parameters

seed BigInteger?

The seed of a fresh Generator(PCG64(SeedSequence(seed))), or null to keep the generator.

options IReadOnlyDictionary<string, object>

Gymnasium's reset options: "low" and "high", anything Python's float() accepts; custom bounds can put the state outside the observation space, as in Gymnasium.

Returns

ResetResult<NDArray>

The float32 observation [cos θ1, sin θ1, cos θ2, sin θ2, θ̇1, θ̇2] and an empty info.

Remarks

The seed is applied before the options are parsed, as super().reset(seed=seed) runs first. Reference: gymnasium/envs/classic_control/acrobot.py.

Exceptions

Error

Thrown for a negative seed, with Gymnasium's message.

ValueError

Thrown for an option float() rejects, or a lower bound above the upper.

OverflowError

Thrown, as Python's OverflowError, for NaN or infinite bounds (from the generator).

DependencyNotInstalled

Thrown in human mode by the first render when no viewer factory is set (DR-20).

Rk4(Func<double[], double[]>, double[], IReadOnlyList<double>)

Gymnasium's module function rk4: integrates dy/dt = derivs(y) from y0 over the sample times t with the classic fourth-order Runge-Kutta step, element by element in float64 in NumPy's order, and returns the first four values of the last sample (Acrobot's state, its torque entry dropped).

public static double[] Rk4(Func<double[], double[]> derivs, double[] y0, IReadOnlyList<double> t)

Parameters

derivs Func<double[], double[]>

The derivative of the system at a point; its result must be as long as y0.

y0 double[]

The initial point; not modified.

t IReadOnlyList<double>

The sample times, at least one; the steps are their differences.

Returns

double[]

The first four values (or fewer, for a shorter system) of the point at the last sample time; a new array.

Remarks

Each stage is two array operations, y0 + h * k, rounded separately, and the combination is y0 + dt / 6.0 * (k1 + 2 * k2 + 2 * k3 + k4) left to right, so the result is NumPy's bit for bit. Reference: gymnasium/envs/classic_control/acrobot.py.

Exceptions

ArgumentNullException

Thrown when an argument is null.

ArgumentException

Thrown when t is empty (NumPy's yout[-1] would raise on the empty result).

ValueError

Thrown when a derivative's length differs from the point's, with NumPy's broadcast message (operands could not be broadcast together with shapes (6,) (5,) ), at the first operation that meets them.

NotSupportedException

Thrown for a one-value derivative of a longer system, which NumPy would broadcast and Gym.NET does not model.

Step(NDArray)

Applies the action's torque to the middle joint for Dt seconds (one Runge-Kutta step), wraps both angles into [-π, π] and bounds both velocities, then terminates when the free end is above the line. Renders in human mode.

public override StepResult<NDArray> Step(NDArray action)

Parameters

action NDArray

An index into AvailTorque: a 0-d integer array (what Discrete(3).Sample() returns), or through the untyped step any .NET integer. Python's list indexing applies, so -3 to 2 are valid. Read, never kept.

Returns

StepResult<NDArray>

The float32 observation, the reward (-1.0, or 0.0 on termination), terminated, truncated=False and {}.

Remarks

Exceptions

AssertionError

Thrown before the first reset (Call reset before using AcrobotEnv object.), whatever the action, as the source asserts first.

TypeError

Thrown for an action Python cannot index a list with, with Python's message (list indices must be integers or slices, not numpy.float64).

IndexError

Thrown for an index outside the list (list index out of range), and for an assigned state of fewer than four values, with NumPy's message.

ValueError

Thrown for an assigned state of more than four values (NumPy's broadcast error).

DependencyNotInstalled

Thrown in human mode by the first render when no viewer factory is set (DR-20).

Wrap(double, double, double)

Gymnasium's module function wrap: moves x into [m, M] by whole periods, one subtraction or addition at a time (so the rounding of each step is Python's, not a remainder's).

public static double Wrap(double x, double m, double M)

Parameters

x double

The value.

m double

The lower end.

M double

The upper end.

Returns

double

The wrapped value; NaN stays NaN.

Remarks

One iteration per period, as in Python: a huge value takes as many iterations as periods it spans, and an infinite one never returns. Reference: gymnasium/envs/classic_control/acrobot.py.