Class CartPoleVectorEnv
- Namespace
- Gym.Environments.Envs.Classic
- Assembly
- Gym.Environments.dll
CartPole vectorized natively: NumEnvs carts stepped together as arrays, with
their own time limit and autoreset (NEXT_STEP), which MakeVec("CartPole-v1", numEnvs: n) builds
through the spec's vector entry point. Byte for byte what Gymnasium's CartPoleVectorEnv computes: the
dynamics of every cart, the generator draws (one (4, num_envs) draw at reset, one (4, k) draw per step
for the k carts that ended at the previous one), the observations and flags, and the frames.
public class CartPoleVectorEnv : VectorEnv<NDArray, NDArray>, IVectorEnv, IDisposable
- Inheritance
-
CartPoleVectorEnv
- Implements
- Inherited Members
- Extension Methods
Remarks
Observations are a (num_envs, 4) float32 array; actions a MultiDiscrete([2, ...]) sample, 0 pushing
left and 1 right. A cart whose episode ended (pole past 12°, cart past 2.4, or max_episode_steps steps) is
re-drawn at the next step, which returns its new state with a reward of 0 and no flags, as Gymnasium's does.
Rewards are a float64 array here (DR-21: Gym.NET's rewards are float64, single or batched) where Gymnasium's are
float32: the values are the same (1, 0, and -1 or -0 with sutton_barto_reward), the dtype is not. Like
Gymnasium's, it renders rgb_array frames only (a list, one per cart) and has no human window. Not
thread-safe.
Reference: gymnasium/envs/classic_control/cartpole.py.
Constructors
CartPoleVectorEnv(CartPoleVectorEnvOptions)
Builds the instance from its options record: the properties set, the constructor's defaults for the rest.
public CartPoleVectorEnv(CartPoleVectorEnvOptions options)
Parameters
optionsCartPoleVectorEnvOptionsThe options.
Exceptions
- ArgumentNullException
Thrown when
optionsis null.
CartPoleVectorEnv(RenderMode, int, long, bool)
Builds the env with the render mode given as the RenderMode enum (DR-84, DR-93), as
CartPoleVectorEnv(int, long, string?, bool)
builds it from the mode's string: new CartPoleVectorEnv(RenderMode.RgbArray) builds what
new CartPoleVectorEnv(renderMode: "rgb_array") builds, the member reaching the env as Gymnasium's string
(Name(RenderMode)), which its RenderMode property then holds.
public CartPoleVectorEnv(RenderMode renderMode, int numEnvs = 1, long maxEpisodeSteps = 500, bool suttonBartoReward = false)
Parameters
renderModeRenderModeThe render mode, which the env must list in its
metadata["render_modes"]as for the string; it behaves exactly as its Gymnasium string does in the other constructor.numEnvsintGymnasium's
num_envs: the number of carts.maxEpisodeStepslongGymnasium's
max_episode_steps: the steps after which a cart's episode is truncated (MakeVecpasses the spec's, 500 for CartPole-v1).suttonBartoRewardboolGymnasium's
sutton_barto_reward: reward -1 at termination and 0 otherwise, instead of 1 per step.
Remarks
The mode is required and comes first, so new CartPoleVectorEnv() keeps meaning the other constructor, without a
mode; leave the mode out there for no rendering. Every other parameter means what it means there.
Exceptions
- ArgumentOutOfRangeException
Thrown for a value RenderMode does not declare (a cast integer such as
(RenderMode)99).
CartPoleVectorEnv(int, long, string?, bool)
Creates the carts.
public CartPoleVectorEnv(int numEnvs = 1, long maxEpisodeSteps = 500, string? renderMode = null, bool suttonBartoReward = false)
Parameters
numEnvsintGymnasium's
num_envs: the number of carts.maxEpisodeStepslongGymnasium's
max_episode_steps: the steps after which a cart's episode is truncated (MakeVecpasses the spec's, 500 for CartPole-v1).renderModestringGymnasium's
render_mode: null, or a mode under which Render() draws every cart (rgb_array).suttonBartoRewardboolGymnasium's
sutton_barto_reward: reward -1 at termination and 0 otherwise, instead of 1 per step.
Properties
ClassMetadata
Gymnasium's class attribute metadata: {"render_modes": ["rgb_array"], "render_fps": 50,
"autoreset_mode": AutoresetMode.NEXT_STEP}. A new dict on every read.
public static Metadata ClassMetadata { get; }
Property Value
ForceMag
Gymnasium's force_mag, 10.0.
public double ForceMag { get; set; }
Property Value
Gravity
Gymnasium's gravity, 9.8.
public double Gravity { get; set; }
Property Value
High
Gymnasium's high: the upper bound of the state draws.
public double High { get; }
Property Value
KinematicsIntegrator
Gymnasium's kinematics_integrator: "euler", or anything else for semi-implicit Euler.
public string KinematicsIntegrator { get; set; }
Property Value
Length
Gymnasium's length, 0.5: actually half the pole's length.
public double Length { get; set; }
Property Value
Low
Gymnasium's low: the lower bound of the state draws, -0.05 or the last reset's options["low"].
public double Low { get; }
Property Value
Masscart
Gymnasium's masscart, 1.0.
public double Masscart { get; set; }
Property Value
Masspole
Gymnasium's masspole, 0.1.
public double Masspole { get; set; }
Property Value
MaxEpisodeSteps
Gymnasium's max_episode_steps: a cart's steps before truncation.
public long MaxEpisodeSteps { get; set; }
Property Value
PolemassLength
Gymnasium's polemass_length: masspole * length, computed once at construction.
public double PolemassLength { get; set; }
Property Value
PrevDone
Gymnasium's prev_done: a new bool array of the carts re-drawn at the next step.
public NDArray PrevDone { get; }
Property Value
ScreenHeight
Gymnasium's screen_height, 400.
public int ScreenHeight { get; set; }
Property Value
ScreenWidth
Gymnasium's screen_width, 600.
public int ScreenWidth { get; set; }
Property Value
Screens
Gymnasium's screens: one surface per cart, created by the first render.
public IReadOnlyList<Surface>? Screens { get; }
Property Value
State
Gymnasium's state: a new (4, num_envs) float64 array of the carts' state (the caller owns it), or
null before the first reset. Assigning an array of that shape injects a state.
public NDArray? State { get; set; }
Property Value
Exceptions
- ValueError
Thrown by the setter for an array of another shape.
Steps
Gymnasium's steps: a new int32 array of each cart's steps in its episode.
public NDArray Steps { get; }
Property Value
StepsBeyondTerminated
Gymnasium's steps_beyond_terminated: always null (Python's None) in the vector env.
public long? StepsBeyondTerminated { get; }
Property Value
- long?
Surf
Gymnasium's surf: the last cart drawn, before it is blitted onto its screen.
public Surface? Surf { get; }
Property Value
SuttonBartoReward
Gymnasium's _sutton_barto_reward.
[PyName("_sutton_barto_reward")]
public bool SuttonBartoReward { get; }
Property Value
Tau
Gymnasium's tau: seconds between state updates, 0.02.
public double Tau { get; set; }
Property Value
ThetaThresholdRadians
Gymnasium's theta_threshold_radians: 12 * 2 * math.pi / 360, the angle that ends an episode.
public double ThetaThresholdRadians { get; set; }
Property Value
TotalMass
Gymnasium's total_mass: masspole + masscart, computed once at construction as in Gymnasium.
public double TotalMass { get; set; }
Property Value
XThreshold
Gymnasium's x_threshold, 2.4: the cart position that ends an episode.
public double XThreshold { get; set; }
Property Value
Methods
CloseExtras()
Gymnasium's close: pygame.quit() once screens exist. There is no window to close here.
protected override void CloseExtras()
Render()
Draws every cart (a list of (400, 600, 3) uint8 frames, one per cart); without a render mode, warns as
Gymnasium does and returns nothing.
public override IReadOnlyList<RenderResult>? Render()
Returns
- IReadOnlyList<RenderResult>
One RenderResult.Rgb per cart (new arrays the caller owns), or null without a render mode.
Remarks
Reference: gymnasium/envs/classic_control/cartpole.py.
Exceptions
- AssertionError
Thrown without a render mode when the env has no spec (
assert self.spec is not None).- ValueError
Thrown before the first reset (
Cartpole's state is None, it probably hasn't be reset yet.), and for a NaN coordinate with pygame's message.- OverflowError
Thrown, as pygame's
OverflowError, for a coordinate pygame cannot draw.
Reset(BigInteger?, IReadOnlyDictionary<string, object?>?)
Draws every cart's state (one uniform(low, high, size=(4, num_envs)) draw) and clears the step counters.
public override VectorResetResult<NDArray> Reset(BigInteger? seed = null, IReadOnlyDictionary<string, object?>? options = null)
Parameters
seedBigInteger?The seed of a fresh generator, or null to keep the stream.
optionsIReadOnlyDictionary<string, object>Optional
low/highbounds of the draws (Gymnasium'smaybe_parse_reset_bounds).
Returns
- VectorResetResult<NDArray>
The
(num_envs, 4)float32 observations (a new array the caller owns) and an empty info.
Remarks
Reference: gymnasium/envs/classic_control/cartpole.py.
Exceptions
- Error
Thrown for a negative seed.
- ValueError
Thrown for bounds
float()rejects or a lower bound above the upper.
Step(NDArray)
Steps every cart with its push (0 left, 1 right), then re-draws the carts that ended at the previous step (one
uniform(low, high, size=(4, k)) draw), zeroing their counters, rewards and flags.
public override VectorStepResult<NDArray> Step(NDArray actions)
Parameters
actionsNDArrayA
(num_envs,)integer array theMultiDiscreteaction space contains.
Returns
- VectorStepResult<NDArray>
The
(num_envs, 4)float32 observations, the float64 rewards, the terminations and truncations, and an empty info; new arrays the caller owns.
Remarks
Reference: gymnasium/envs/classic_control/cartpole.py.
Exceptions
- AssertionError
Thrown for actions the space does not contain (
array([0, 2]) (<class 'numpy.ndarray'>) invalid), or before the first reset (Call reset before using step method.).