Class AcrobotEnv
- Namespace
- Gym.Environments.Envs.Classic
- Assembly
- Gym.Environments.dll
Acrobot-v1: two links joined in a chain, hanging from a fixed joint, with a torque of -1, 0 or +1 (actions 0, 1, 2) on the joint between them; the goal is to swing the free end above a line one link-length over the base. -1 per step until then, 0 on the terminating step. Byte for byte what Gymnasium's Acrobot computes, errors included.
public class AcrobotEnv : Env<NDArray, NDArray>, IEnv, IDisposable, IDynamicAttributes
- Inheritance
-
AcrobotEnv
- Implements
- Inherited Members
- Extension Methods
Remarks
The dynamics are Gymnasium's: the "book" equations (BookOrNips) integrated over Dt by one
fourth-order Runge-Kutta step, in float64, every product and sum in the source's order and every x ** 2
the C runtime's pow, as NumPy computes them. The reset state is float32 (uniform(...).astype(np.float32)),
so the first observation's cosines and sines are NumPy's own float32 ones (through NumSharp, bit for bit) and the
first step starts from the float32-rounded values; after a step the state is float64.
The action is not checked against the action space, as in Gymnasium: it indexes AvailTorque the way
a Python list is indexed, so -1, -2 and -3 are valid (the torques of 2, 1 and 0), and other values raise Python's
IndexError or TypeError. The physical constants are Gymnasium's class attributes, settable per env
here (Python's are shared by the class); RLPy's unused action_arrow, domain_fig and
actions_num are not carried.
Rendering is Gymnasium's (the line, the links and joints, pixel for pixel). Human mode renders into the window from
viewer_factory or Gymnasium.HumanViewerFactory (DR-20) at 15 frames per second. A state angle that
is infinite never wraps, in Gymnasium's wrap as here, so such a step does not return.
Not thread-safe. Reference: gymnasium/envs/classic_control/acrobot.py.
Its info is always {}, as Gymnasium's, so it has no typed info record (DR-95).
Constructors
AcrobotEnv(AcrobotEnvOptions)
Builds the instance from its options record: the properties set, the constructor's defaults for the rest.
public AcrobotEnv(AcrobotEnvOptions options)
Parameters
optionsAcrobotEnvOptionsThe options.
Exceptions
- ArgumentNullException
Thrown when
optionsis null.
AcrobotEnv(RenderMode, EnvViewerFactory?)
Builds the env with the render mode given as the RenderMode enum (DR-84, DR-93), as
AcrobotEnv(string?, EnvViewerFactory?)
builds it from the mode's string: new AcrobotEnv(RenderMode.RgbArray) builds what
new AcrobotEnv(renderMode: "rgb_array") builds, the member reaching the env as Gymnasium's string
(Name(RenderMode)), which its RenderMode property then holds.
public AcrobotEnv(RenderMode renderMode, EnvViewerFactory? viewerFactory = null)
Parameters
renderModeRenderModeThe render mode, which the env must list in its
metadata["render_modes"]as for the string; it behaves exactly as its Gymnasium string does in the other constructor.viewerFactoryEnvViewerFactoryGym.NET's
viewer_factory(DR-20) for human mode; null usesGymnasium.HumanViewerFactory.
Remarks
The mode is required and comes first, so new AcrobotEnv() keeps meaning the other constructor, without a
mode; leave the mode out there for no rendering. Every other parameter means what it means there.
Exceptions
- ArgumentOutOfRangeException
Thrown for a value RenderMode does not declare (a cast integer such as
(RenderMode)99).
AcrobotEnv(string?, EnvViewerFactory?)
Creates the env, stateless until the first reset.
public AcrobotEnv(string? renderMode = null, EnvViewerFactory? viewerFactory = null)
Parameters
renderModestringGymnasium's
render_mode: null,"human"or"rgb_array"; not validated, as in Gymnasium (makechecks it against the metadata).viewerFactoryEnvViewerFactoryGym.NET's
viewer_factory(DR-20) for human mode; null usesGymnasium.HumanViewerFactory.
Properties
AvailTorque
Gymnasium's AVAIL_TORQUE, [-1.0, 0.0, +1]: the torque of each action, indexed as a Python list (a
negative action counts from the end). Python's last entry is the int 1, which joins the float64 state as
1.0, so a double holds it exactly.
[PyName("AVAIL_TORQUE")]
public double[] AvailTorque { get; set; }
Property Value
- double[]
BookOrNips
Gymnasium's book_or_nips, "book": which equation gives the second joint's acceleration. "nips"
uses the NIPS paper's, which leaves out a velocity term; any other string the book's (and the Java
implementation's).
public string BookOrNips { get; set; }
Property Value
ClassMetadata
Gymnasium's class attribute metadata: {"render_modes": ["human", "rgb_array"], "render_fps": 15}.
Every env starts with a copy as its instance metadata.
public static Metadata ClassMetadata { get; }
Property Value
Remarks
A new dict on every read, so editing one env's metadata never reaches another (Python's is one shared dict).
Clock
Gymnasium's clock: paces human rendering, null until the first render.
public Clock? Clock { get; }
Property Value
Dt
Gymnasium's dt, 0.2 s: the integration time step, one Runge-Kutta step per env step.
public double Dt { get; set; }
Property Value
IsOpen
[PyName("isopen")]
public bool IsOpen { get; set; }
Property Value
LinkComPos1
Gymnasium's LINK_COM_POS_1, 0.5 m: the first link's center of mass.
[PyName("LINK_COM_POS_1")]
public double LinkComPos1 { get; set; }
Property Value
LinkComPos2
Gymnasium's LINK_COM_POS_2, 0.5 m: the second link's center of mass.
[PyName("LINK_COM_POS_2")]
public double LinkComPos2 { get; set; }
Property Value
LinkLength1
Gymnasium's LINK_LENGTH_1, 1.0 m. Also sizes the frame (the world spans both links plus 0.2).
[PyName("LINK_LENGTH_1")]
public double LinkLength1 { get; set; }
Property Value
LinkLength2
Gymnasium's LINK_LENGTH_2, 1.0 m. Used only by rendering, as in Gymnasium's dynamics.
[PyName("LINK_LENGTH_2")]
public double LinkLength2 { get; set; }
Property Value
LinkMass1
Gymnasium's LINK_MASS_1, 1.0 kg.
[PyName("LINK_MASS_1")]
public double LinkMass1 { get; set; }
Property Value
LinkMass2
Gymnasium's LINK_MASS_2, 1.0 kg.
[PyName("LINK_MASS_2")]
public double LinkMass2 { get; set; }
Property Value
LinkMoi
Gymnasium's LINK_MOI, 1.0: both links' moment of inertia.
[PyName("LINK_MOI")]
public double LinkMoi { get; set; }
Property Value
MaxVel1
Gymnasium's MAX_VEL_1, 4π: the first joint's angular velocity bound, applied every step.
[PyName("MAX_VEL_1")]
public double MaxVel1 { get; set; }
Property Value
MaxVel2
Gymnasium's MAX_VEL_2, 9π: the second joint's angular velocity bound.
[PyName("MAX_VEL_2")]
public double MaxVel2 { get; set; }
Property Value
Screen
Gymnasium's screen: the surface every frame is blitted onto, null until the first render.
Close() keeps it, as Python's close does.
public Surface? Screen { get; }
Property Value
ScreenDim
Gymnasium's SCREEN_DIM, 500: the frame's width and height in pixels.
[PyName("SCREEN_DIM")]
public int ScreenDim { get; set; }
Property Value
State
Gymnasium's state, [θ1, θ2, θ̇1, θ̇2]: a new array of the values on every read, null
before the first reset. Assigning injects a state, as env.state = np.array([...]) does: float64 values of
any length, checked by the next step with Python's errors; null is Python's None.
public double[]? State { get; set; }
Property Value
- double[]
Remarks
Python's array is float32 after a reset and float64 after a step; the values read here are exactly its values, a float32 one widened.
TorqueNoiseMax
Gymnasium's torque_noise_max, 0.0: when positive, every step adds uniform(-max, max) to the torque,
one draw from the env's generator.
public double TorqueNoiseMax { get; set; }
Property Value
Methods
Bound(double, double, double)
Gymnasium's module function bound with scalar limits: min(max(x, m), M) with Python's built-ins,
which keep x when a comparison is false (so NaN stays NaN).
public static double Bound(double x, double m, double M)
Parameters
Returns
- double
The bounded value.
Remarks
Reference: gymnasium/envs/classic_control/acrobot.py.
Close()
Once something was rendered, shuts the human-mode window down and marks the env closed (IsOpen). Like Gymnasium's, it keeps Screen, so a later render reuses it.
public override void Close()
Remarks
Reference: gymnasium/envs/classic_control/acrobot.py.
ConvertAction(object?)
The untyped step's action conversion, with the source's order kept: its state assert first, then the list index
on the action as given, so the errors name the Python counterpart of the value given (not float for a
double, not numpy.float32 for a float).
protected override NDArray ConvertAction(object? action)
Parameters
actionobjectThe action as given to the untyped step.
Returns
- NDArray
The action as a 0-d integer array: the one given, or a new one the step releases.
Exceptions
- AssertionError
Thrown before the first reset.
- TypeError
Thrown for a value Python cannot index a list with.
- IndexError
Thrown for an index outside the list, or an integer beyond 64 bits.
Render()
Draws Gymnasium's frame with the same primitives in the same order: the target line, then each link as a rotated rectangle with a circle at its joint, in a y-up world flipped at the end.
public override RenderResult Render()
Returns
- RenderResult
rgb_array: the(500, 500, 3)uint8 frame, a new array the caller owns.human, any other mode, and every mode before the first reset: nothing.
Remarks
After a reset the state is float32, so the first link's end and both angles are computed in float32 (NumPy's float32 cosine and sine), as in Python; after a step, in float64. Reference: gymnasium/envs/classic_control/acrobot.py.
Exceptions
- AssertionError
Thrown without a render mode when the env has no spec (
assert self.spec is not None).- IndexError
Thrown for an assigned state of fewer than two values, with NumPy's message.
- KeyError
Thrown in human mode when
metadata["render_fps"]was removed.- ValueError
Thrown for a NaN coordinate (a NaN state), with pygame's or Python's message.
- OverflowError
Thrown, as pygame's
OverflowError, for a coordinate pygame cannot draw.- DependencyNotInstalled
Thrown on the first human render when no viewer factory is set (DR-20).
Reset(BigInteger?, IReadOnlyDictionary<string, object?>?)
Starts an episode: reseeds the generator when a seed is given, then draws the four state variables from
uniform(low, high, size=(4,)) (default ±0.1, or the "low"/"high" options) and rounds them to
float32. Renders in human mode.
public override ResetResult<NDArray> Reset(BigInteger? seed = null, IReadOnlyDictionary<string, object?>? options = null)
Parameters
seedBigInteger?The seed of a fresh
Generator(PCG64(SeedSequence(seed))), or null to keep the generator.optionsIReadOnlyDictionary<string, object>Gymnasium's reset options:
"low"and"high", anything Python'sfloat()accepts; custom bounds can put the state outside the observation space, as in Gymnasium.
Returns
- ResetResult<NDArray>
The float32 observation
[cos θ1, sin θ1, cos θ2, sin θ2, θ̇1, θ̇2]and an empty info.
Remarks
The seed is applied before the options are parsed, as super().reset(seed=seed) runs first. Reference:
gymnasium/envs/classic_control/acrobot.py.
Exceptions
- Error
Thrown for a negative seed, with Gymnasium's message.
- ValueError
Thrown for an option
float()rejects, or a lower bound above the upper.- OverflowError
Thrown, as Python's
OverflowError, for NaN or infinite bounds (from the generator).- DependencyNotInstalled
Thrown in human mode by the first render when no viewer factory is set (DR-20).
Rk4(Func<double[], double[]>, double[], IReadOnlyList<double>)
Gymnasium's module function rk4: integrates dy/dt = derivs(y) from y0 over the
sample times t with the classic fourth-order Runge-Kutta step, element by element in float64 in
NumPy's order, and returns the first four values of the last sample (Acrobot's state, its torque entry dropped).
public static double[] Rk4(Func<double[], double[]> derivs, double[] y0, IReadOnlyList<double> t)
Parameters
derivsFunc<double[], double[]>The derivative of the system at a point; its result must be as long as
y0.y0double[]The initial point; not modified.
tIReadOnlyList<double>The sample times, at least one; the steps are their differences.
Returns
- double[]
The first four values (or fewer, for a shorter system) of the point at the last sample time; a new array.
Remarks
Each stage is two array operations, y0 + h * k, rounded separately, and the combination is
y0 + dt / 6.0 * (k1 + 2 * k2 + 2 * k3 + k4) left to right, so the result is NumPy's bit for bit. Reference:
gymnasium/envs/classic_control/acrobot.py.
Exceptions
- ArgumentNullException
Thrown when an argument is null.
- ArgumentException
Thrown when
tis empty (NumPy'syout[-1]would raise on the empty result).- ValueError
Thrown when a derivative's length differs from the point's, with NumPy's broadcast message (
operands could not be broadcast together with shapes (6,) (5,)), at the first operation that meets them.- NotSupportedException
Thrown for a one-value derivative of a longer system, which NumPy would broadcast and Gym.NET does not model.
Step(NDArray)
Applies the action's torque to the middle joint for Dt seconds (one Runge-Kutta step), wraps both angles into [-π, π] and bounds both velocities, then terminates when the free end is above the line. Renders in human mode.
public override StepResult<NDArray> Step(NDArray action)
Parameters
actionNDArrayAn index into AvailTorque: a 0-d integer array (what
Discrete(3).Sample()returns), or through the untyped step any .NET integer. Python's list indexing applies, so -3 to 2 are valid. Read, never kept.
Returns
- StepResult<NDArray>
The float32 observation, the reward (-1.0, or 0.0 on termination),
terminated,truncated=Falseand{}.
Remarks
Reference: gymnasium/envs/classic_control/acrobot.py.
Exceptions
- AssertionError
Thrown before the first reset (
Call reset before using AcrobotEnv object.), whatever the action, as the source asserts first.- TypeError
Thrown for an action Python cannot index a list with, with Python's message (
list indices must be integers or slices, not numpy.float64).- IndexError
Thrown for an index outside the list (
list index out of range), and for an assigned state of fewer than four values, with NumPy's message.- ValueError
Thrown for an assigned state of more than four values (NumPy's broadcast error).
- DependencyNotInstalled
Thrown in human mode by the first render when no viewer factory is set (DR-20).
Wrap(double, double, double)
Gymnasium's module function wrap: moves x into [m, M]
by whole periods, one subtraction or addition at a time (so the rounding of each step is Python's, not a
remainder's).
public static double Wrap(double x, double m, double M)
Parameters
Returns
- double
The wrapped value; NaN stays NaN.
Remarks
One iteration per period, as in Python: a huge value takes as many iterations as periods it spans, and an infinite one never returns. Reference: gymnasium/envs/classic_control/acrobot.py.