Table of Contents

Class CartPoleEnv

Namespace
Gym.Environments.Envs.Classic
Assembly
Gym.Environments.dll

CartPole-v1: a pole hinged on a cart that is pushed left (action 0) or right (action 1) with a fixed force, until the pole leans more than 12° or the cart leaves ±2.4. Byte for byte what Gymnasium's CartPole computes: the same float64 dynamics in the same operation order, float32 observations, rewards, termination, warnings, errors, reset options, rgb_array pixels and human-mode pacing.

public class CartPoleEnv : Env<NDArray, NDArray>, IEnv, IDisposable, IDynamicAttributes
Inheritance
CartPoleEnv
Implements
Inherited Members
Extension Methods

Remarks

The physics parameters are Gymnasium's instance attributes, as settable properties (Gravity, Tau, KinematicsIntegrator, ...): changing one after construction has the effect it has in Python, and like Python the derived ones (TotalMass, PolemassLength) are computed once in the constructor and not recomputed. State is Gymnasium's self.state, a float64 (x, x_dot, theta, theta_dot); the observation is always a fresh float32 copy of it.

The render mode is fixed at construction, as in Gymnasium: "rgb_array" makes Render() return the frame; "human" renders into a viewer after every Reset(BigInteger?, IReadOnlyDictionary<string, object?>?) and Step(NDArray), paced at metadata["render_fps"] (50) by a Clock, and returns nothing. The viewer comes from the factory given to the constructor (viewer_factory), else from Gymnasium.HumanViewerFactory; with neither, the first human render raises DependencyNotInstalled, where Gymnasium raises it for a missing pygame (DR-20). Pass Factory for human mode without a window.

The unwrapped env never truncates: TimeLimit adds the 500-step limit of CartPole-v1, as make does. Not thread-safe: step, reset and render an instance from one thread at a time.

Reference: gymnasium/envs/classic_control/cartpole.py.

Its info is always {}, as Gymnasium's, so it has no typed info record (DR-95).

Constructors

CartPoleEnv(EnvViewerFactory?)

0.2's new CartPoleEnv(viewerFactory), kept so 0.2 code compiles until 2.0 (DR-58). 0.2 drew only when Render() was called, and its headless runs passed Factory: that factory, or null, builds the env without a render mode (headless and unpaced), and any other factory builds a human-mode env drawing into the window it creates.

[Obsolete("0.2's constructor: write new CartPoleEnv(renderMode: \"human\", viewerFactory: factory), or new CartPoleEnv() headless. Removed in 2.0 with Gym.Legacy.")]
public CartPoleEnv(EnvViewerFactory? viewerFactory)

Parameters

viewerFactory EnvViewerFactory

The viewer factory, as 0.2 took it.

Remarks

Footguns: a human-mode env draws every Step(NDArray) itself, paced at 50 frames a second, so 0.2's Render() after each step draws every frame twice (drop the call); a headless env's Render() warns once, as Gymnasium's does without a render mode, and returns nothing. new CartPoleEnv(null) is ambiguous next to the IEnvViewer constructor; cast the null.

CartPoleEnv(CartPoleEnvOptions)

Builds the instance from its options record: the properties set, the constructor's defaults for the rest.

public CartPoleEnv(CartPoleEnvOptions options)

Parameters

options CartPoleEnvOptions

The options.

Exceptions

ArgumentNullException

Thrown when options is null.

CartPoleEnv(IEnvViewer, bool)

Creates a human-mode env rendering into an existing viewer.

public CartPoleEnv(IEnvViewer viewer, bool suttonBartoReward = false)

Parameters

viewer IEnvViewer

The viewer, Gymnasium's display window.

suttonBartoReward bool

Gymnasium's sutton_barto_reward (see CartPoleEnv(bool, string?, EnvViewerFactory?)).

Exceptions

ArgumentNullException

Thrown when viewer is null.

CartPoleEnv(RenderMode, bool, EnvViewerFactory?)

Builds the env with the render mode given as the RenderMode enum (DR-84, DR-93), as CartPoleEnv(bool, string?, EnvViewerFactory?) builds it from the mode's string: new CartPoleEnv(RenderMode.RgbArray) builds what new CartPoleEnv(renderMode: "rgb_array") builds, the member reaching the env as Gymnasium's string (Name(RenderMode)), which its RenderMode property then holds.

public CartPoleEnv(RenderMode renderMode, bool suttonBartoReward = false, EnvViewerFactory? viewerFactory = null)

Parameters

renderMode RenderMode

The render mode, which the env must list in its metadata["render_modes"] as for the string; it behaves exactly as its Gymnasium string does in the other constructor.

suttonBartoReward bool

Gymnasium's sutton_barto_reward: true rewards 0 per surviving step and -1 on and after termination (Sutton and Barto's original), instead of 1 per step including the terminating one.

viewerFactory EnvViewerFactory

Gym.NET's viewer_factory (DR-20): creates the human-mode window on the first human render; null uses Gymnasium.HumanViewerFactory, read at that render.

Remarks

The mode is required and comes first, so new CartPoleEnv() keeps meaning the other constructor, without a mode; leave the mode out there for no rendering. Every other parameter means what it means there.

Exceptions

ArgumentOutOfRangeException

Thrown for a value RenderMode does not declare (a cast integer such as (RenderMode)99).

CartPoleEnv(bool, string?, EnvViewerFactory?)

Creates CartPole-v1 with Gymnasium's attributes, action space Discrete(2) and observation space Box(-high, high, float32) with high = [4.8, inf, 0.41887903, inf]. The generator is created lazily from OS entropy; seed it with Reset(seed: s).

public CartPoleEnv(bool suttonBartoReward = false, string? renderMode = null, EnvViewerFactory? viewerFactory = null)

Parameters

suttonBartoReward bool

Gymnasium's sutton_barto_reward: true rewards 0 per surviving step and -1 on and after termination (Sutton and Barto's original), instead of 1 per step including the terminating one.

renderMode string

Gymnasium's render_mode: null (no rendering), "human" or "rgb_array". Not validated, as in Gymnasium (make checks it against metadata["render_modes"]): another mode draws each frame off-screen and Render() returns nothing, as Python's does.

viewerFactory EnvViewerFactory

Gym.NET's viewer_factory (DR-20): creates the human-mode window on the first human render; null uses Gymnasium.HumanViewerFactory, read at that render.

Properties

ClassMetadata

Gymnasium's class attribute metadata, which make reads before building the env: {"render_modes": ["human", "rgb_array"], "render_fps": 50}, keys in that order. Every env starts with a copy of it as its instance Metadata.

public static Metadata ClassMetadata { get; }

Property Value

Metadata

Remarks

A new dict on every read, so editing one env's metadata never reaches another (Python's is one shared dict).

Clock

Gymnasium's clock: paces human rendering, null until the first render.

public Clock? Clock { get; }

Property Value

Clock

ForceMag

Gymnasium's force_mag, 10.0 N: the push of either action.

public double ForceMag { get; set; }

Property Value

double

Gravity

Gymnasium's gravity, 9.8 m/s².

public double Gravity { get; set; }

Property Value

double

IsOpen

Gymnasium's isopen: true until Close() shuts a render window down.

[PyName("isopen")]
public bool IsOpen { get; set; }

Property Value

bool

KinematicsIntegrator

Gymnasium's kinematics_integrator: "euler" (the default) updates positions from the old velocities; any other string selects semi-implicit Euler, which updates velocities first.

public string KinematicsIntegrator { get; set; }

Property Value

string

Length

Gymnasium's length, 0.5 m: half the pole's length.

public double Length { get; set; }

Property Value

double

Masscart

Gymnasium's masscart, 1.0 kg. Changing it does not update TotalMass, as in Python.

public double Masscart { get; set; }

Property Value

double

Masspole

Gymnasium's masspole, 0.1 kg. Changing it does not update TotalMass or PolemassLength.

public double Masspole { get; set; }

Property Value

double

PolemassLength

Gymnasium's polemass_length, masspole * length as computed at construction (0.05).

public double PolemassLength { get; set; }

Property Value

double

Screen

Gymnasium's screen: the surface every frame is blitted onto (the display window's in human mode), null until the first render. Close() keeps it, as Python's close does.

public Surface? Screen { get; }

Property Value

Surface

ScreenHeight

Gymnasium's screen_height, 400 pixels; read when a frame is rendered.

public int ScreenHeight { get; set; }

Property Value

int

ScreenWidth

Gymnasium's screen_width, 600 pixels; read when a frame is rendered.

public int ScreenWidth { get; set; }

Property Value

int

State

Gymnasium's state: [x, x_dot, theta, theta_dot] in float64, null before the first reset. Assigning it (state injection) makes the next step start from there; Step(NDArray) replaces it with a new array, so a reference read earlier keeps its values.

public double[]? State { get; set; }

Property Value

double[]

StepsBeyondTerminated

Gymnasium's steps_beyond_terminated: null while the pole is up, then 0 on the terminating step and one more for every step after it.

public int? StepsBeyondTerminated { get; set; }

Property Value

int?

Surf

Gymnasium's surf: the last frame drawn before it was blitted onto Screen, or null.

public Surface? Surf { get; }

Property Value

Surface

SuttonBartoReward

Gymnasium's _sutton_barto_reward, fixed at construction.

[PyName("_sutton_barto_reward")]
public bool SuttonBartoReward { get; }

Property Value

bool

Tau

Gymnasium's tau, 0.02 s between state updates. A float64: 0.02f moves the state at step 1 (B-11).

public double Tau { get; set; }

Property Value

double

ThetaThresholdRadians

Gymnasium's theta_threshold_radians, 12 * 2 * pi / 360: the angle (±12°) that ends an episode.

public double ThetaThresholdRadians { get; set; }

Property Value

double

TotalMass

Gymnasium's total_mass, masspole + masscart as computed at construction (1.1).

public double TotalMass { get; set; }

Property Value

double

XThreshold

Gymnasium's x_threshold, 2.4: the cart position that ends an episode.

public double XThreshold { get; set; }

Property Value

double

Methods

Close()

Once something was rendered, shuts the human-mode window down and marks the env closed (IsOpen). Like Gymnasium's, it keeps Screen, so a later rgb_array render reuses it; a later human render opens a new window from the factory.

public override void Close()

Remarks

ConvertAction(object?)

The untyped step's action conversion, with CartPole's own assert first: an action that is not an element of Discrete(2) fails with Python's message for exactly the value given (np.int64(5) (<class 'numpy.int64'>) invalid, 'x' (<class 'str'>) invalid).

protected override NDArray ConvertAction(object? action)

Parameters

action object

The action as given to Step(object?): any integer, a bool, an enum member, or a 0-d integer array.

Returns

NDArray

The action as a 0-d array: the one given, or a new int64 one for a plain integer.

Exceptions

AssertionError

Thrown for anything Discrete(2).contains rejects.

Render()

Draws the frame Gymnasium's CartPole draws with pygame, with the same primitives in the same order (Gym.Rendering reproduces SDL_gfx), so the rgb_array pixels are Gymnasium's exactly. The track is at y = 100 in a y-up world, flipped vertically at the end.

public override RenderResult Render()

Returns

RenderResult

rgb_array: the (400, 600, 3) uint8 frame, a new array the caller owns. human: nothing, after showing the frame in the viewer at render_fps. Any other mode, and every mode before the first reset: nothing, as Gymnasium returns None.

Remarks

Without a render mode, Gymnasium asserts the env has a spec and warns once (You are calling render method without specifying any render mode. ...), naming the spec's id; an env built with its constructor has no spec, so it raises the bare AssertionError, as Python's does. Reference: gymnasium/envs/classic_control/cartpole.py.

Exceptions

AssertionError

Thrown without a render mode when the env has no spec (assert self.spec is not None).

KeyError

Thrown in human mode when metadata["render_fps"] was removed.

ValueError

Thrown for a NaN coordinate (a NaN state), with pygame's message.

OverflowError

Thrown, as pygame's OverflowError, for a coordinate pygame cannot draw: beyond a C long in a polygon (x = 1e10), or beyond Sint16 in the axle's circle (x = 300), or an infinite one.

Reset(BigInteger?, IReadOnlyDictionary<string, object?>?)

Starts an episode: reseeds NpRandom when a seed is given, then draws the four state variables from uniform(low, high, size=(4,)), with low/high from the options (default ±0.05), and clears StepsBeyondTerminated. Renders in human mode.

public override ResetResult<NDArray> Reset(BigInteger? seed = null, IReadOnlyDictionary<string, object?>? options = null)

Parameters

seed BigInteger?

The seed of a fresh Generator(PCG64(SeedSequence(seed))), or null to keep the current generator.

options IReadOnlyDictionary<string, object>

Gymnasium's reset options: "low" and "high", anything Python's float() accepts; custom bounds can put the state outside the observation space, as in Gymnasium.

Returns

ResetResult<NDArray>

The float32 observation and an empty info, (obs, {}).

Remarks

The seed is applied before the options are parsed, as super().reset(seed=seed) runs first: a reset that fails on its options still leaves the freshly seeded generator, undrawn. Reference: gymnasium/envs/classic_control/cartpole.py.

Exceptions

Error

Thrown for a negative seed, with Gymnasium's message.

ValueError

Thrown for an option float() rejects, or a lower bound above the upper (see MaybeParseResetBounds(IReadOnlyDictionary<string, object?>?, double, double)).

OverflowError

Thrown, as Python's OverflowError, when the bounds are NaN or infinite (high - low range exceeds valid bounds, from the generator) or an integer bound is beyond the float range.

Step(NDArray)

Pushes the cart left (0) or right (1) with ForceMag and integrates one step of Tau seconds with Gymnasium's equations, operation for operation in float64. Renders in human mode.

public override StepResult<NDArray> Step(NDArray action)

Parameters

action NDArray

An element of Discrete(2): a 0-d integer array holding 0 or 1, what ActionSpace.Sample() returns (DR-08). The untyped Step(object?) also takes a plain .NET integer. Read during the step, never kept.

Returns

StepResult<NDArray>

The float32 observation, the reward (1.0 per step and on termination, 0.0 after it; 0.0, -1.0 and -1.0 with SuttonBartoReward), terminated, truncated=False and {}.

Remarks

Stepping after termination warns once through Warn(string, WarningCategory, string, int), as Gymnasium does, and keeps going. Reference: gymnasium/envs/classic_control/cartpole.py.

Exceptions

AssertionError

Thrown for an action outside the action space, with Python's message (2 (<class 'int'>) invalid), and before the first reset (Call reset before using step method.), in that order.

ValueError

Thrown when an assigned State does not hold exactly four values (Python's unpacking error).