Table of Contents

Class CartPoleVectorEnv

Namespace
Gym.Environments.Envs.Classic
Assembly
Gym.Environments.dll

CartPole vectorized natively: NumEnvs carts stepped together as arrays, with their own time limit and autoreset (NEXT_STEP), which MakeVec("CartPole-v1", numEnvs: n) builds through the spec's vector entry point. Byte for byte what Gymnasium's CartPoleVectorEnv computes: the dynamics of every cart, the generator draws (one (4, num_envs) draw at reset, one (4, k) draw per step for the k carts that ended at the previous one), the observations and flags, and the frames.

public class CartPoleVectorEnv : VectorEnv<NDArray, NDArray>, IVectorEnv, IDisposable
Inheritance
CartPoleVectorEnv
Implements
Inherited Members
Extension Methods

Remarks

Observations are a (num_envs, 4) float32 array; actions a MultiDiscrete([2, ...]) sample, 0 pushing left and 1 right. A cart whose episode ended (pole past 12°, cart past 2.4, or max_episode_steps steps) is re-drawn at the next step, which returns its new state with a reward of 0 and no flags, as Gymnasium's does.

Rewards are a float64 array here (DR-21: Gym.NET's rewards are float64, single or batched) where Gymnasium's are float32: the values are the same (1, 0, and -1 or -0 with sutton_barto_reward), the dtype is not. Like Gymnasium's, it renders rgb_array frames only (a list, one per cart) and has no human window. Not thread-safe.

Reference: gymnasium/envs/classic_control/cartpole.py.

Constructors

CartPoleVectorEnv(CartPoleVectorEnvOptions)

Builds the instance from its options record: the properties set, the constructor's defaults for the rest.

public CartPoleVectorEnv(CartPoleVectorEnvOptions options)

Parameters

options CartPoleVectorEnvOptions

The options.

Exceptions

ArgumentNullException

Thrown when options is null.

CartPoleVectorEnv(RenderMode, int, long, bool)

Builds the env with the render mode given as the RenderMode enum (DR-84, DR-93), as CartPoleVectorEnv(int, long, string?, bool) builds it from the mode's string: new CartPoleVectorEnv(RenderMode.RgbArray) builds what new CartPoleVectorEnv(renderMode: "rgb_array") builds, the member reaching the env as Gymnasium's string (Name(RenderMode)), which its RenderMode property then holds.

public CartPoleVectorEnv(RenderMode renderMode, int numEnvs = 1, long maxEpisodeSteps = 500, bool suttonBartoReward = false)

Parameters

renderMode RenderMode

The render mode, which the env must list in its metadata["render_modes"] as for the string; it behaves exactly as its Gymnasium string does in the other constructor.

numEnvs int

Gymnasium's num_envs: the number of carts.

maxEpisodeSteps long

Gymnasium's max_episode_steps: the steps after which a cart's episode is truncated (MakeVec passes the spec's, 500 for CartPole-v1).

suttonBartoReward bool

Gymnasium's sutton_barto_reward: reward -1 at termination and 0 otherwise, instead of 1 per step.

Remarks

The mode is required and comes first, so new CartPoleVectorEnv() keeps meaning the other constructor, without a mode; leave the mode out there for no rendering. Every other parameter means what it means there.

Exceptions

ArgumentOutOfRangeException

Thrown for a value RenderMode does not declare (a cast integer such as (RenderMode)99).

CartPoleVectorEnv(int, long, string?, bool)

Creates the carts.

public CartPoleVectorEnv(int numEnvs = 1, long maxEpisodeSteps = 500, string? renderMode = null, bool suttonBartoReward = false)

Parameters

numEnvs int

Gymnasium's num_envs: the number of carts.

maxEpisodeSteps long

Gymnasium's max_episode_steps: the steps after which a cart's episode is truncated (MakeVec passes the spec's, 500 for CartPole-v1).

renderMode string

Gymnasium's render_mode: null, or a mode under which Render() draws every cart (rgb_array).

suttonBartoReward bool

Gymnasium's sutton_barto_reward: reward -1 at termination and 0 otherwise, instead of 1 per step.

Properties

ClassMetadata

Gymnasium's class attribute metadata: {"render_modes": ["rgb_array"], "render_fps": 50, "autoreset_mode": AutoresetMode.NEXT_STEP}. A new dict on every read.

public static Metadata ClassMetadata { get; }

Property Value

Metadata

ForceMag

Gymnasium's force_mag, 10.0.

public double ForceMag { get; set; }

Property Value

double

Gravity

Gymnasium's gravity, 9.8.

public double Gravity { get; set; }

Property Value

double

High

Gymnasium's high: the upper bound of the state draws.

public double High { get; }

Property Value

double

KinematicsIntegrator

Gymnasium's kinematics_integrator: "euler", or anything else for semi-implicit Euler.

public string KinematicsIntegrator { get; set; }

Property Value

string

Length

Gymnasium's length, 0.5: actually half the pole's length.

public double Length { get; set; }

Property Value

double

Low

Gymnasium's low: the lower bound of the state draws, -0.05 or the last reset's options["low"].

public double Low { get; }

Property Value

double

Masscart

Gymnasium's masscart, 1.0.

public double Masscart { get; set; }

Property Value

double

Masspole

Gymnasium's masspole, 0.1.

public double Masspole { get; set; }

Property Value

double

MaxEpisodeSteps

Gymnasium's max_episode_steps: a cart's steps before truncation.

public long MaxEpisodeSteps { get; set; }

Property Value

long

PolemassLength

Gymnasium's polemass_length: masspole * length, computed once at construction.

public double PolemassLength { get; set; }

Property Value

double

PrevDone

Gymnasium's prev_done: a new bool array of the carts re-drawn at the next step.

public NDArray PrevDone { get; }

Property Value

NDArray

ScreenHeight

Gymnasium's screen_height, 400.

public int ScreenHeight { get; set; }

Property Value

int

ScreenWidth

Gymnasium's screen_width, 600.

public int ScreenWidth { get; set; }

Property Value

int

Screens

Gymnasium's screens: one surface per cart, created by the first render.

public IReadOnlyList<Surface>? Screens { get; }

Property Value

IReadOnlyList<Surface>

State

Gymnasium's state: a new (4, num_envs) float64 array of the carts' state (the caller owns it), or null before the first reset. Assigning an array of that shape injects a state.

public NDArray? State { get; set; }

Property Value

NDArray

Exceptions

ValueError

Thrown by the setter for an array of another shape.

Steps

Gymnasium's steps: a new int32 array of each cart's steps in its episode.

public NDArray Steps { get; }

Property Value

NDArray

StepsBeyondTerminated

Gymnasium's steps_beyond_terminated: always null (Python's None) in the vector env.

public long? StepsBeyondTerminated { get; }

Property Value

long?

Surf

Gymnasium's surf: the last cart drawn, before it is blitted onto its screen.

public Surface? Surf { get; }

Property Value

Surface

SuttonBartoReward

Gymnasium's _sutton_barto_reward.

[PyName("_sutton_barto_reward")]
public bool SuttonBartoReward { get; }

Property Value

bool

Tau

Gymnasium's tau: seconds between state updates, 0.02.

public double Tau { get; set; }

Property Value

double

ThetaThresholdRadians

Gymnasium's theta_threshold_radians: 12 * 2 * math.pi / 360, the angle that ends an episode.

public double ThetaThresholdRadians { get; set; }

Property Value

double

TotalMass

Gymnasium's total_mass: masspole + masscart, computed once at construction as in Gymnasium.

public double TotalMass { get; set; }

Property Value

double

XThreshold

Gymnasium's x_threshold, 2.4: the cart position that ends an episode.

public double XThreshold { get; set; }

Property Value

double

Methods

CloseExtras()

Gymnasium's close: pygame.quit() once screens exist. There is no window to close here.

protected override void CloseExtras()

Render()

Draws every cart (a list of (400, 600, 3) uint8 frames, one per cart); without a render mode, warns as Gymnasium does and returns nothing.

public override IReadOnlyList<RenderResult>? Render()

Returns

IReadOnlyList<RenderResult>

One RenderResult.Rgb per cart (new arrays the caller owns), or null without a render mode.

Remarks

Exceptions

AssertionError

Thrown without a render mode when the env has no spec (assert self.spec is not None).

ValueError

Thrown before the first reset (Cartpole's state is None, it probably hasn't be reset yet.), and for a NaN coordinate with pygame's message.

OverflowError

Thrown, as pygame's OverflowError, for a coordinate pygame cannot draw.

Reset(BigInteger?, IReadOnlyDictionary<string, object?>?)

Draws every cart's state (one uniform(low, high, size=(4, num_envs)) draw) and clears the step counters.

public override VectorResetResult<NDArray> Reset(BigInteger? seed = null, IReadOnlyDictionary<string, object?>? options = null)

Parameters

seed BigInteger?

The seed of a fresh generator, or null to keep the stream.

options IReadOnlyDictionary<string, object>

Optional low/high bounds of the draws (Gymnasium's maybe_parse_reset_bounds).

Returns

VectorResetResult<NDArray>

The (num_envs, 4) float32 observations (a new array the caller owns) and an empty info.

Remarks

Exceptions

Error

Thrown for a negative seed.

ValueError

Thrown for bounds float() rejects or a lower bound above the upper.

Step(NDArray)

Steps every cart with its push (0 left, 1 right), then re-draws the carts that ended at the previous step (one uniform(low, high, size=(4, k)) draw), zeroing their counters, rewards and flags.

public override VectorStepResult<NDArray> Step(NDArray actions)

Parameters

actions NDArray

A (num_envs,) integer array the MultiDiscrete action space contains.

Returns

VectorStepResult<NDArray>

The (num_envs, 4) float32 observations, the float64 rewards, the terminations and truncations, and an empty info; new arrays the caller owns.

Remarks

Exceptions

AssertionError

Thrown for actions the space does not contain (array([0, 2]) (<class 'numpy.ndarray'>) invalid), or before the first reset (Call reset before using step method.).