Class CliffWalkingEnv
- Namespace
- Gym.Environments.Envs.ToyText
- Assembly
- Gym.Environments.dll
CliffWalking-v1 and CliffWalkingSlippery-v1: a 4x12 grid walked from the bottom-left corner to the bottom-right one
along a cliff; stepping off it costs -100 and returns the agent to the start, every other step costs -1. Slippery
moves go perpendicular with probability 2/3. Byte for byte what Gymnasium's CliffWalking computes: the transition
table P, the generator draws, observations, rewards, {"prob": p} infos, ansi text and
rgb_array pixels.
public class CliffWalkingEnv : Env<NDArray, NDArray>, IEnv, IDisposable, IDynamicAttributes
- Inheritance
-
CliffWalkingEnv
- Implements
- Inherited Members
- Extension Methods
Remarks
Like Gymnasium's, the env checks no action against its space: self.P[self.s][a] is a dict lookup, so any
number equal to 0 (up), 1 (right), 2 (down) or 3 (left) is an action, anything else raises KeyError. Each
step draws one float64 from the env's generator, even on a certain move. The goal is not absorbing: a step from it
moves on, as in Gymnasium. Observations are 0-d int64 arrays (DR-08); rewards are Gymnasium's -1 and -100 as
float64.
Rendering is Gymnasium's pixel art at 64-pixel cells (its art scale is 1, so the images keep their sizes);
ansi marks the agent x, the goal T, the cliff C. Any render mode other than
human and ansi draws and returns the frame, as Gymnasium's else branches do. Human mode renders
into the window from viewer_factory or Gymnasium.HumanViewerFactory (DR-20) at 4 frames per second.
Not thread-safe.
Reference: gymnasium/envs/toy_text/cliffwalking.py.
Its info reads typed (DR-91): CliffWalkingInfo info = step.Info; (CliffWalkingInfo), with the dict's own values still behind the indexer.
Constructors
CliffWalkingEnv(CliffWalkingEnvOptions)
Builds the instance from its options record: the properties set, the constructor's defaults for the rest.
public CliffWalkingEnv(CliffWalkingEnvOptions options)
Parameters
optionsCliffWalkingEnvOptionsThe options.
Exceptions
- ArgumentNullException
Thrown when
optionsis null.
CliffWalkingEnv(RenderMode, bool, EnvViewerFactory?)
Builds the env with the render mode given as the RenderMode enum (DR-84, DR-93), as
CliffWalkingEnv(string?, bool, EnvViewerFactory?)
builds it from the mode's string: new CliffWalkingEnv(RenderMode.RgbArray) builds what
new CliffWalkingEnv(renderMode: "rgb_array") builds, the member reaching the env as Gymnasium's string
(Name(RenderMode)), which its RenderMode property then holds.
public CliffWalkingEnv(RenderMode renderMode, bool isSlippery = false, EnvViewerFactory? viewerFactory = null)
Parameters
renderModeRenderModeThe render mode, which the env must list in its
metadata["render_modes"]as for the string; it behaves exactly as its Gymnasium string does in the other constructor.isSlipperyboolGymnasium's
is_slippery: each move goes as intended or to either side, with probability 1/3 each.viewerFactoryEnvViewerFactoryGym.NET's
viewer_factory(DR-20) for human mode; null usesGymnasium.HumanViewerFactory.
Remarks
The mode is required and comes first, so new CliffWalkingEnv() keeps meaning the other constructor, without a
mode; leave the mode out there for no rendering. Every other parameter means what it means there.
Exceptions
- ArgumentOutOfRangeException
Thrown for a value RenderMode does not declare (a cast integer such as
(RenderMode)99).
CliffWalkingEnv(string?, bool, EnvViewerFactory?)
Creates the env.
public CliffWalkingEnv(string? renderMode = null, bool isSlippery = false, EnvViewerFactory? viewerFactory = null)
Parameters
renderModestringGymnasium's
render_mode: null,"human","rgb_array"or"ansi".isSlipperyboolGymnasium's
is_slippery: each move goes as intended or to either side, with probability 1/3 each.viewerFactoryEnvViewerFactoryGym.NET's
viewer_factory(DR-20) for human mode; null usesGymnasium.HumanViewerFactory.
Fields
Down
Gymnasium's DOWN, action 2.
public const long Down = 2
Field Value
Left
Gymnasium's LEFT, action 3.
public const long Left = 3
Field Value
Right
Gymnasium's RIGHT, action 1.
public const long Right = 1
Field Value
Up
Gymnasium's UP, action 0.
public const long Up = 0
Field Value
Properties
AgentImg
Gymnasium's agent_img.
public Surface? AgentImg { get; }
Property Value
CellSize
Gymnasium's cell_size: (64, 64) pixels.
public (int Width, int Height) CellSize { get; }
Property Value
ClassMetadata
Gymnasium's class attribute metadata: {"render_modes": ["human", "rgb_array", "ansi"], "render_fps": 4}.
Every env starts with a copy as its instance metadata.
public static Metadata ClassMetadata { get; }
Property Value
Remarks
A new dict on every read, so editing one env's metadata never reaches another (Python's is one shared dict).
CliffImgs
Gymnasium's cliff_imgs: the cliff tiles by name (left, mid_a, mid_b, right).
public IReadOnlyDictionary<string, Surface>? CliffImgs { get; }
Property Value
Clock
Gymnasium's clock: paces human rendering, null until the first image render.
public Clock? Clock { get; }
Property Value
GoalFlagImg
Gymnasium's goal_flag_img.
public Surface? GoalFlagImg { get; }
Property Value
GoalImg
Gymnasium's goal_img, the letter G.
public Surface? GoalImg { get; }
Property Value
GroundImg
Gymnasium's ground_img; null until the first image render.
public Surface? GroundImg { get; }
Property Value
InitialStateDistrib
Gymnasium's initial_state_distrib: 1 at StartStateIndex, a new array on every read.
public double[] InitialStateDistrib { get; }
Property Value
- double[]
IsSlippery
Gymnasium's is_slippery, fixed at construction.
public bool IsSlippery { get; }
Property Value
LastAction
Gymnasium's lastaction: the action key of the last step, or null after a reset (Python's None).
[PyName("lastaction")]
public long? LastAction { get; }
Property Value
- long?
NA
Gymnasium's nA: 4 actions.
[PyName("nA")]
public int NA { get; }
Property Value
NS
Gymnasium's nS: 48 states.
[PyName("nS")]
public int NS { get; }
Property Value
P
Gymnasium's P: for each state and action, the possible outcomes (probability, next state, reward,
terminated) in Gymnasium's order (a slippery move: counter-clockwise, as intended, clockwise).
[PyName("P")]
public IReadOnlyDictionary<long, IReadOnlyDictionary<long, IReadOnlyList<ToyTextTransition>>> P { get; }
Property Value
S
Gymnasium's s: the agent's state, row * 12 + col; null before the first reset.
Assigning one injects a state (one outside the grid makes the next step's lookup fail with KeyError).
[PyName("s")]
public long? S { get; set; }
Property Value
- long?
Shape
Gymnasium's shape: (4, 12).
public (int Rows, int Columns) Shape { get; }
Property Value
StartImg
Gymnasium's start_img, the letter S.
public Surface? StartImg { get; }
Property Value
StartStateIndex
Gymnasium's start_state_index: 36, the cell (3, 0).
public long StartStateIndex { get; }
Property Value
WindowSize
Gymnasium's window_size: (768, 256) pixels, twelve cells by four.
public (int Width, int Height) WindowSize { get; }
Property Value
WindowSurface
Gymnasium's window_surface: the frame every image render draws on, null until the first one.
public Surface? WindowSurface { get; }
Property Value
Methods
Close()
Once a window surface exists, shuts the human-mode window down (pygame's display.quit()); the surface is kept.
public override void Close()
Remarks
Reference: gymnasium/envs/toy_text/cliffwalking.py.
ConvertAction(object?)
The untyped step's action conversion, with Python's order kept: the state's existence first, then the dict lookup on the action as given, so the errors name the Python counterpart of the value given.
protected override NDArray ConvertAction(object? action)
Parameters
actionobjectThe action as given to the untyped step.
Returns
- NDArray
The action as an array: the one given, or a new one the step releases.
Exceptions
- AttributeError
Thrown before the first reset.
- KeyError
Thrown for an action no key equals.
- TypeError
Thrown for an array of one or more dimensions.
Render()
Renders by the render mode: ansi returns the grid as text; any other mode draws Gymnasium's pixel art
(and returns the frame unless the mode is human).
public override RenderResult Render()
Returns
- RenderResult
ansi: the text.human: nothing. Any other mode: the(256, 768, 3)uint8 frame, a new array the caller owns.
Remarks
Reference: gymnasium/envs/toy_text/cliffwalking.py.
Exceptions
- AssertionError
Thrown without a render mode when the env has no spec (
assert self.spec is not None).- AttributeError
Thrown before the first reset (no
s; for images after the first cell's ground was drawn).- KeyError
Thrown in human mode when
metadata["render_fps"]was removed.- DependencyNotInstalled
Thrown on the first human render when no viewer factory is set (DR-20).
Reset(BigInteger?, IReadOnlyDictionary<string, object?>?)
Starts an episode: reseeds the generator when a seed is given, then draws the start state (one draw, always state 36) and clears the last action. Renders in human mode.
public override ResetResult<NDArray> Reset(BigInteger? seed = null, IReadOnlyDictionary<string, object?>? options = null)
Parameters
seedBigInteger?The seed of a fresh
Generator(PCG64(SeedSequence(seed))), or null to keep the generator.optionsIReadOnlyDictionary<string, object>Ignored, as in Gymnasium.
Returns
- ResetResult<NDArray>
The state as a 0-d int64 array, and
{"prob": 1}(a Python int).
Remarks
Reference: gymnasium/envs/toy_text/cliffwalking.py.
Exceptions
- Error
Thrown for a negative seed, with Gymnasium's message.
- DependencyNotInstalled
Thrown in human mode by the first render when no viewer factory is set (DR-20).
Step(NDArray)
Moves by the action: looks up P[s][a], draws one outcome (one float64 from the generator) and moves there.
Renders in human mode.
public override StepResult<NDArray> Step(NDArray action)
Parameters
actionNDArrayA number equal to 0 (up), 1 (right), 2 (down) or 3 (left): a 0-d array, or through the untyped step any .NET number.
Returns
- StepResult<NDArray>
The new state (a 0-d int64 array), the reward (-1, or -100 off the cliff), whether it is the goal,
truncated=False, and{"prob": p}.
Remarks
Reference: gymnasium/envs/toy_text/cliffwalking.py.
Exceptions
- AttributeError
Thrown before the first reset (
'CliffWalkingEnv' object has no attribute 's'), before the action is looked at.- KeyError
Thrown for an action no key equals, or an injected state outside the grid, with Python's message.
- TypeError
Thrown for an array action of one or more dimensions (
unhashable type: 'numpy.ndarray').- DependencyNotInstalled
Thrown in human mode by the first render when no viewer factory is set (DR-20).