Class CartPoleEnv
- Namespace
- Gym.Environments.Envs.Classic
- Assembly
- Gym.Environments.dll
CartPole-v1: a pole hinged on a cart that is pushed left (action 0) or right (action 1) with a fixed force, until
the pole leans more than 12° or the cart leaves ±2.4. Byte for byte what Gymnasium's CartPole computes: the same
float64 dynamics in the same operation order, float32 observations, rewards, termination, warnings, errors, reset
options, rgb_array pixels and human-mode pacing.
public class CartPoleEnv : Env<NDArray, NDArray>, IEnv, IDisposable, IDynamicAttributes
- Inheritance
-
CartPoleEnv
- Implements
- Inherited Members
- Extension Methods
Remarks
The physics parameters are Gymnasium's instance attributes, as settable properties (Gravity,
Tau, KinematicsIntegrator, ...): changing one after construction has the effect it has
in Python, and like Python the derived ones (TotalMass, PolemassLength) are computed
once in the constructor and not recomputed. State is Gymnasium's self.state, a float64
(x, x_dot, theta, theta_dot); the observation is always a fresh float32 copy of it.
The render mode is fixed at construction, as in Gymnasium: "rgb_array" makes Render() return
the frame; "human" renders into a viewer after every Reset(BigInteger?, IReadOnlyDictionary<string, object?>?) and Step(NDArray), paced at
metadata["render_fps"] (50) by a Clock, and returns nothing. The viewer comes from the
factory given to the constructor (viewer_factory), else from Gymnasium.HumanViewerFactory; with
neither, the first human render raises DependencyNotInstalled, where Gymnasium raises it for a
missing pygame (DR-20). Pass Factory for human mode without a window.
The unwrapped env never truncates: TimeLimit adds the 500-step limit of CartPole-v1, as make does.
Not thread-safe: step, reset and render an instance from one thread at a time.
Reference: gymnasium/envs/classic_control/cartpole.py.
Its info is always {}, as Gymnasium's, so it has no typed info record (DR-95).
Constructors
CartPoleEnv(EnvViewerFactory?)
0.2's new CartPoleEnv(viewerFactory), kept so 0.2 code compiles until 2.0 (DR-58). 0.2 drew only when
Render() was called, and its headless runs passed Factory: that factory, or
null, builds the env without a render mode (headless and unpaced), and any other factory builds
a human-mode env drawing into the window it creates.
[Obsolete("0.2's constructor: write new CartPoleEnv(renderMode: \"human\", viewerFactory: factory), or new CartPoleEnv() headless. Removed in 2.0 with Gym.Legacy.")]
public CartPoleEnv(EnvViewerFactory? viewerFactory)
Parameters
viewerFactoryEnvViewerFactoryThe viewer factory, as 0.2 took it.
Remarks
Footguns: a human-mode env draws every Step(NDArray) itself, paced at 50 frames a second, so 0.2's
Render() after each step draws every frame twice (drop the call); a headless env's Render() warns
once, as Gymnasium's does without a render mode, and returns nothing. new CartPoleEnv(null) is ambiguous
next to the IEnvViewer constructor; cast the null.
CartPoleEnv(CartPoleEnvOptions)
Builds the instance from its options record: the properties set, the constructor's defaults for the rest.
public CartPoleEnv(CartPoleEnvOptions options)
Parameters
optionsCartPoleEnvOptionsThe options.
Exceptions
- ArgumentNullException
Thrown when
optionsis null.
CartPoleEnv(IEnvViewer, bool)
Creates a human-mode env rendering into an existing viewer.
public CartPoleEnv(IEnvViewer viewer, bool suttonBartoReward = false)
Parameters
viewerIEnvViewerThe viewer, Gymnasium's display window.
suttonBartoRewardboolGymnasium's
sutton_barto_reward(see CartPoleEnv(bool, string?, EnvViewerFactory?)).
Exceptions
- ArgumentNullException
Thrown when
vieweris null.
CartPoleEnv(RenderMode, bool, EnvViewerFactory?)
Builds the env with the render mode given as the RenderMode enum (DR-84, DR-93), as
CartPoleEnv(bool, string?, EnvViewerFactory?)
builds it from the mode's string: new CartPoleEnv(RenderMode.RgbArray) builds what
new CartPoleEnv(renderMode: "rgb_array") builds, the member reaching the env as Gymnasium's string
(Name(RenderMode)), which its RenderMode property then holds.
public CartPoleEnv(RenderMode renderMode, bool suttonBartoReward = false, EnvViewerFactory? viewerFactory = null)
Parameters
renderModeRenderModeThe render mode, which the env must list in its
metadata["render_modes"]as for the string; it behaves exactly as its Gymnasium string does in the other constructor.suttonBartoRewardboolGymnasium's
sutton_barto_reward: true rewards 0 per surviving step and -1 on and after termination (Sutton and Barto's original), instead of 1 per step including the terminating one.viewerFactoryEnvViewerFactoryGym.NET's
viewer_factory(DR-20): creates the human-mode window on the first human render; null usesGymnasium.HumanViewerFactory, read at that render.
Remarks
The mode is required and comes first, so new CartPoleEnv() keeps meaning the other constructor, without a
mode; leave the mode out there for no rendering. Every other parameter means what it means there.
Exceptions
- ArgumentOutOfRangeException
Thrown for a value RenderMode does not declare (a cast integer such as
(RenderMode)99).
CartPoleEnv(bool, string?, EnvViewerFactory?)
Creates CartPole-v1 with Gymnasium's attributes, action space Discrete(2) and observation space
Box(-high, high, float32) with high = [4.8, inf, 0.41887903, inf]. The generator is created lazily
from OS entropy; seed it with Reset(seed: s).
public CartPoleEnv(bool suttonBartoReward = false, string? renderMode = null, EnvViewerFactory? viewerFactory = null)
Parameters
suttonBartoRewardboolGymnasium's
sutton_barto_reward: true rewards 0 per surviving step and -1 on and after termination (Sutton and Barto's original), instead of 1 per step including the terminating one.renderModestringGymnasium's
render_mode: null (no rendering),"human"or"rgb_array". Not validated, as in Gymnasium (makechecks it againstmetadata["render_modes"]): another mode draws each frame off-screen and Render() returns nothing, as Python's does.viewerFactoryEnvViewerFactoryGym.NET's
viewer_factory(DR-20): creates the human-mode window on the first human render; null usesGymnasium.HumanViewerFactory, read at that render.
Properties
ClassMetadata
Gymnasium's class attribute metadata, which make reads before building the env:
{"render_modes": ["human", "rgb_array"], "render_fps": 50}, keys in that order. Every env starts with a
copy of it as its instance Metadata.
public static Metadata ClassMetadata { get; }
Property Value
Remarks
A new dict on every read, so editing one env's metadata never reaches another (Python's is one shared dict).
Clock
Gymnasium's clock: paces human rendering, null until the first render.
public Clock? Clock { get; }
Property Value
ForceMag
Gymnasium's force_mag, 10.0 N: the push of either action.
public double ForceMag { get; set; }
Property Value
Gravity
Gymnasium's gravity, 9.8 m/s².
public double Gravity { get; set; }
Property Value
IsOpen
[PyName("isopen")]
public bool IsOpen { get; set; }
Property Value
KinematicsIntegrator
Gymnasium's kinematics_integrator: "euler" (the default) updates positions from the old velocities;
any other string selects semi-implicit Euler, which updates velocities first.
public string KinematicsIntegrator { get; set; }
Property Value
Length
Gymnasium's length, 0.5 m: half the pole's length.
public double Length { get; set; }
Property Value
Masscart
Gymnasium's masscart, 1.0 kg. Changing it does not update TotalMass, as in Python.
public double Masscart { get; set; }
Property Value
Masspole
Gymnasium's masspole, 0.1 kg. Changing it does not update TotalMass or PolemassLength.
public double Masspole { get; set; }
Property Value
PolemassLength
Gymnasium's polemass_length, masspole * length as computed at construction (0.05).
public double PolemassLength { get; set; }
Property Value
Screen
Gymnasium's screen: the surface every frame is blitted onto (the display window's in human mode),
null until the first render. Close() keeps it, as Python's close does.
public Surface? Screen { get; }
Property Value
ScreenHeight
Gymnasium's screen_height, 400 pixels; read when a frame is rendered.
public int ScreenHeight { get; set; }
Property Value
ScreenWidth
Gymnasium's screen_width, 600 pixels; read when a frame is rendered.
public int ScreenWidth { get; set; }
Property Value
State
Gymnasium's state: [x, x_dot, theta, theta_dot] in float64, null before the first
reset. Assigning it (state injection) makes the next step start from there; Step(NDArray) replaces it with a
new array, so a reference read earlier keeps its values.
public double[]? State { get; set; }
Property Value
- double[]
StepsBeyondTerminated
Gymnasium's steps_beyond_terminated: null while the pole is up, then 0 on the
terminating step and one more for every step after it.
public int? StepsBeyondTerminated { get; set; }
Property Value
- int?
Surf
public Surface? Surf { get; }
Property Value
SuttonBartoReward
Gymnasium's _sutton_barto_reward, fixed at construction.
[PyName("_sutton_barto_reward")]
public bool SuttonBartoReward { get; }
Property Value
Tau
Gymnasium's tau, 0.02 s between state updates. A float64: 0.02f moves the state at step 1 (B-11).
public double Tau { get; set; }
Property Value
ThetaThresholdRadians
Gymnasium's theta_threshold_radians, 12 * 2 * pi / 360: the angle (±12°) that ends an episode.
public double ThetaThresholdRadians { get; set; }
Property Value
TotalMass
Gymnasium's total_mass, masspole + masscart as computed at construction (1.1).
public double TotalMass { get; set; }
Property Value
XThreshold
Gymnasium's x_threshold, 2.4: the cart position that ends an episode.
public double XThreshold { get; set; }
Property Value
Methods
Close()
Once something was rendered, shuts the human-mode window down and marks the env closed (IsOpen).
Like Gymnasium's, it keeps Screen, so a later rgb_array render reuses it; a later human
render opens a new window from the factory.
public override void Close()
Remarks
Reference: gymnasium/envs/classic_control/cartpole.py.
ConvertAction(object?)
The untyped step's action conversion, with CartPole's own assert first: an action that is not an element of
Discrete(2) fails with Python's message for exactly the value given (np.int64(5) (<class
'numpy.int64'>) invalid, 'x' (<class 'str'>) invalid).
protected override NDArray ConvertAction(object? action)
Parameters
actionobjectThe action as given to Step(object?): any integer, a bool, an enum member, or a 0-d integer array.
Returns
- NDArray
The action as a 0-d array: the one given, or a new int64 one for a plain integer.
Exceptions
- AssertionError
Thrown for anything
Discrete(2).containsrejects.
Render()
Draws the frame Gymnasium's CartPole draws with pygame, with the same primitives in the same order (Gym.Rendering
reproduces SDL_gfx), so the rgb_array pixels are Gymnasium's exactly. The track is at y = 100 in a
y-up world, flipped vertically at the end.
public override RenderResult Render()
Returns
- RenderResult
rgb_array: the(400, 600, 3)uint8 frame, a new array the caller owns.human: nothing, after showing the frame in the viewer atrender_fps. Any other mode, and every mode before the first reset: nothing, as Gymnasium returnsNone.
Remarks
Without a render mode, Gymnasium asserts the env has a spec and warns once (You are calling render method
without specifying any render mode. ...), naming the spec's id; an env built with its constructor has no
spec, so it raises the bare AssertionError, as Python's does. Reference: gymnasium/envs/classic_control/cartpole.py.
Exceptions
- AssertionError
Thrown without a render mode when the env has no spec (
assert self.spec is not None).- KeyError
Thrown in human mode when
metadata["render_fps"]was removed.- ValueError
Thrown for a NaN coordinate (a NaN state), with pygame's message.
- OverflowError
Thrown, as pygame's
OverflowError, for a coordinate pygame cannot draw: beyond a C long in a polygon (x = 1e10), or beyond Sint16 in the axle's circle (x = 300), or an infinite one.
Reset(BigInteger?, IReadOnlyDictionary<string, object?>?)
Starts an episode: reseeds NpRandom when a seed is given, then draws the four
state variables from uniform(low, high, size=(4,)), with low/high from the options (default
±0.05), and clears StepsBeyondTerminated. Renders in human mode.
public override ResetResult<NDArray> Reset(BigInteger? seed = null, IReadOnlyDictionary<string, object?>? options = null)
Parameters
seedBigInteger?The seed of a fresh
Generator(PCG64(SeedSequence(seed))), or null to keep the current generator.optionsIReadOnlyDictionary<string, object>Gymnasium's reset options:
"low"and"high", anything Python'sfloat()accepts; custom bounds can put the state outside the observation space, as in Gymnasium.
Returns
- ResetResult<NDArray>
The float32 observation and an empty info,
(obs, {}).
Remarks
The seed is applied before the options are parsed, as super().reset(seed=seed) runs first: a reset that
fails on its options still leaves the freshly seeded generator, undrawn. Reference: gymnasium/envs/classic_control/cartpole.py.
Exceptions
- Error
Thrown for a negative seed, with Gymnasium's message.
- ValueError
Thrown for an option
float()rejects, or a lower bound above the upper (see MaybeParseResetBounds(IReadOnlyDictionary<string, object?>?, double, double)).- OverflowError
Thrown, as Python's
OverflowError, when the bounds are NaN or infinite (high - low range exceeds valid bounds, from the generator) or an integer bound is beyond the float range.
Step(NDArray)
Pushes the cart left (0) or right (1) with ForceMag and integrates one step of Tau seconds with Gymnasium's equations, operation for operation in float64. Renders in human mode.
public override StepResult<NDArray> Step(NDArray action)
Parameters
actionNDArrayAn element of
Discrete(2): a 0-d integer array holding 0 or 1, whatActionSpace.Sample()returns (DR-08). The untyped Step(object?) also takes a plain .NET integer. Read during the step, never kept.
Returns
- StepResult<NDArray>
The float32 observation, the reward (1.0 per step and on termination, 0.0 after it; 0.0, -1.0 and -1.0 with SuttonBartoReward),
terminated,truncated=Falseand{}.
Remarks
Stepping after termination warns once through Warn(string, WarningCategory, string, int), as Gymnasium does, and keeps going. Reference: gymnasium/envs/classic_control/cartpole.py.
Exceptions
- AssertionError
Thrown for an action outside the action space, with Python's message (
2 (<class 'int'>) invalid), and before the first reset (Call reset before using step method.), in that order.- ValueError
Thrown when an assigned State does not hold exactly four values (Python's unpacking error).