Table of Contents

Class FrozenLakeEnv

Namespace
Gym.Environments.Envs.ToyText
Assembly
Gym.Environments.dll

FrozenLake-v1 and FrozenLake8x8-v1: walk from the start S to the goal G across a frozen lake without falling into a hole H; on slippery ice a move goes perpendicular with probability 1 - success_rate. Byte for byte what Gymnasium's FrozenLake computes: the transition table P, the generator draws, observations, rewards, {"prob": p} infos, ansi text and rgb_array pixels.

public class FrozenLakeEnv : Env<NDArray, NDArray>, IEnv, IDisposable, IDynamicAttributes
Inheritance
FrozenLakeEnv
Implements
Inherited Members
Extension Methods

Remarks

Like Gymnasium's, the env checks no action against its space: self.P[self.s][a] is a dict lookup, so any number equal to 0, 1, 2 or 3 is an action (True, 2.0, np.float32(1.0)) and anything else raises KeyError with the action's repr. Each step draws one float64 from the env's generator (CategoricalSample(IReadOnlyList<double>, Generator)), even on solid ice. Observations are 0-d int64 arrays (DR-08), the Python int Gymnasium returns; rewards are reward_schedule's values as float64.

The map is held as Gymnasium's S1 character array (Desc, built by CharMap(IReadOnlyList<string>)). Rendering is Gymnasium's pixel art: tiles scaled to the cell, overlays scaled by round and blitted at truncated positions, a one-pixel grid; ansi marks the agent in red with Gymnasium's escape codes. Human mode renders into the window from viewer_factory or Gymnasium.HumanViewerFactory (DR-20) at 4 frames per second.

Accepted deviations: reward_schedule is a list of doubles, and null means its default (1, 0, 0) (Python's None would fail in min); LastAction keeps the key the action found, where Python keeps the action object (the ansi caption still applies Python's list-index rules to the action given). Not thread-safe. Reference: gymnasium/envs/toy_text/frozen_lake.py.

Its info reads typed (DR-91): FrozenLakeInfo info = step.Info; (FrozenLakeInfo), with the dict's own values still behind the indexer.

Constructors

FrozenLakeEnv(FrozenLakeEnvOptions)

Builds the instance from its options record: the properties set, the constructor's defaults for the rest.

public FrozenLakeEnv(FrozenLakeEnvOptions options)

Parameters

options FrozenLakeEnvOptions

The options.

Exceptions

ArgumentNullException

Thrown when options is null.

FrozenLakeEnv(RenderMode, IReadOnlyList<string>?, string?, bool, double, IReadOnlyList<double>?, EnvViewerFactory?)

Builds the env with the render mode given as the RenderMode enum (DR-84, DR-93), as FrozenLakeEnv(string?, IReadOnlyList<string>?, string?, bool, double, IReadOnlyList<double>?, EnvViewerFactory?) builds it from the mode's string: new FrozenLakeEnv(RenderMode.RgbArray) builds what new FrozenLakeEnv(renderMode: "rgb_array") builds, the member reaching the env as Gymnasium's string (Name(RenderMode)), which its RenderMode property then holds.

public FrozenLakeEnv(RenderMode renderMode, IReadOnlyList<string>? desc = null, string? mapName = "4x4", bool isSlippery = true, double successRate = 0.3333333333333333, IReadOnlyList<double>? rewardSchedule = null, EnvViewerFactory? viewerFactory = null)

Parameters

renderMode RenderMode

The render mode, which the env must list in its metadata["render_modes"] as for the string; it behaves exactly as its Gymnasium string does in the other constructor.

desc IReadOnlyList<string>

Gymnasium's desc: the map's rows (S start, F frozen, H hole, G goal), all of one length.

mapName string

Gymnasium's map_name: "4x4" or "8x8" (Maps), used when desc is null; with both null, GenerateRandomMap(int, double, BigInteger?) makes an 8x8 map from fresh entropy (not reproducible).

isSlippery bool

Gymnasium's is_slippery: moves may slide perpendicular, each with probability (1 - success_rate) / 2.

successRate double

Gymnasium's success_rate: the probability of moving as intended on slippery ice, 1/3 by default.

rewardSchedule IReadOnlyList<double>

Gymnasium's reward_schedule: the rewards for reaching the goal, a hole and any other tile; null is (1, 0, 0).

viewerFactory EnvViewerFactory

Gym.NET's viewer_factory (DR-20) for human mode; null uses Gymnasium.HumanViewerFactory.

Remarks

The mode is required and comes first, so new FrozenLakeEnv() keeps meaning the other constructor, without a mode; leave the mode out there for no rendering. Every other parameter means what it means there.

Exceptions

ArgumentOutOfRangeException

Thrown for a value RenderMode does not declare (a cast integer such as (RenderMode)99).

KeyError

Thrown for a mapName Maps does not hold, with Python's message ('5x5').

ValueError

Thrown for rows of different lengths, with NumPy's message, or an empty rewardSchedule (min() arg is an empty sequence).

ZeroDivisionError

Thrown, as Python's ZeroDivisionError, for a map without rows or columns (the cell size divides by them).

FrozenLakeEnv(string?, IReadOnlyList<string>?, string?, bool, double, IReadOnlyList<double>?, EnvViewerFactory?)

Creates the env; the map comes from desc, else mapName, else a random 8x8 map.

public FrozenLakeEnv(string? renderMode = null, IReadOnlyList<string>? desc = null, string? mapName = "4x4", bool isSlippery = true, double successRate = 0.3333333333333333, IReadOnlyList<double>? rewardSchedule = null, EnvViewerFactory? viewerFactory = null)

Parameters

renderMode string

Gymnasium's render_mode: null, "human", "ansi" or "rgb_array".

desc IReadOnlyList<string>

Gymnasium's desc: the map's rows (S start, F frozen, H hole, G goal), all of one length.

mapName string

Gymnasium's map_name: "4x4" or "8x8" (Maps), used when desc is null; with both null, GenerateRandomMap(int, double, BigInteger?) makes an 8x8 map from fresh entropy (not reproducible).

isSlippery bool

Gymnasium's is_slippery: moves may slide perpendicular, each with probability (1 - success_rate) / 2.

successRate double

Gymnasium's success_rate: the probability of moving as intended on slippery ice, 1/3 by default.

rewardSchedule IReadOnlyList<double>

Gymnasium's reward_schedule: the rewards for reaching the goal, a hole and any other tile; null is (1, 0, 0).

viewerFactory EnvViewerFactory

Gym.NET's viewer_factory (DR-20) for human mode; null uses Gymnasium.HumanViewerFactory.

Exceptions

KeyError

Thrown for a mapName Maps does not hold, with Python's message ('5x5').

ValueError

Thrown for rows of different lengths, with NumPy's message, or an empty rewardSchedule (min() arg is an empty sequence).

ZeroDivisionError

Thrown, as Python's ZeroDivisionError, for a map without rows or columns (the cell size divides by them).

Fields

Down

Gymnasium's DOWN, action 1.

public const long Down = 1

Field Value

long

Left

Gymnasium's LEFT, action 0.

public const long Left = 0

Field Value

long

Right

Gymnasium's RIGHT, action 2.

public const long Right = 2

Field Value

long

Up

Gymnasium's UP, action 3.

public const long Up = 3

Field Value

long

Properties

AgentImg

Gymnasium's agent_img, scaled from its 40x36 art by the cell's smaller side over 64.

public Surface? AgentImg { get; }

Property Value

Surface

CellSize

Gymnasium's cell_size: window_size // (ncol, nrow) pixels.

public (int Width, int Height) CellSize { get; }

Property Value

(int Width, int Height)

ClassMetadata

Gymnasium's class attribute metadata: {"render_modes": ["human", "ansi", "rgb_array"], "render_fps": 4}. Every env starts with a copy as its instance metadata.

public static Metadata ClassMetadata { get; }

Property Value

Metadata

Remarks

A new dict on every read, so editing one env's metadata never reaches another (Python's is one shared dict).

Clock

Gymnasium's clock: paces human rendering, null until the first render.

public Clock? Clock { get; }

Property Value

Clock

Desc

Gymnasium's desc: the map as NumPy's (nrow, ncol) S1 array, a new array on every read that the caller owns.

public NDArray Desc { get; }

Property Value

NDArray

GoalFlagImg

Gymnasium's goal_flag_img, scaled from its 24x32 art.

public Surface? GoalFlagImg { get; }

Property Value

Surface

HoleImg

Gymnasium's hole_img, scaled to the cell.

public Surface? HoleImg { get; }

Property Value

Surface

IceCrackImgs

Gymnasium's ice_crack_imgs, the two crack overlays scaled to the cell.

public IReadOnlyList<Surface>? IceCrackImgs { get; }

Property Value

IReadOnlyList<Surface>

IceImg

Gymnasium's ice_img, scaled to the cell; null until the first image render.

public Surface? IceImg { get; }

Property Value

Surface

InitialStateDistrib

Gymnasium's initial_state_distrib: the start tiles' uniform distribution over the states, a new array on every read.

public double[] InitialStateDistrib { get; }

Property Value

double[]

LastAction

Gymnasium's lastaction: the action key of the last step, or null after a reset (Python's None).

[PyName("lastaction")]
public long? LastAction { get; }

Property Value

long?

LetterGImg

Gymnasium's letter_g_img, scaled from its 12x20 art.

public Surface? LetterGImg { get; }

Property Value

Surface

LetterSImg

Gymnasium's letter_s_img, scaled from its 12x20 art.

public Surface? LetterSImg { get; }

Property Value

Surface

Maps

Gymnasium's MAPS: the named maps, "4x4" and "8x8", row by row.

public static IReadOnlyDictionary<string, IReadOnlyList<string>> Maps { get; }

Property Value

IReadOnlyDictionary<string, IReadOnlyList<string>>

Ncol

Gymnasium's ncol: the map's columns.

public int Ncol { get; }

Property Value

int

Nrow

Gymnasium's nrow: the map's rows.

public int Nrow { get; }

Property Value

int

P

Gymnasium's P: for each state and action, the possible outcomes (probability, next state, reward, terminated), in Gymnasium's order (for a slippery move: the move counter-clockwise, as intended, clockwise).

[PyName("P")]
public IReadOnlyDictionary<long, IReadOnlyDictionary<long, IReadOnlyList<ToyTextTransition>>> P { get; }

Property Value

IReadOnlyDictionary<long, IReadOnlyDictionary<long, IReadOnlyList<ToyTextTransition>>>

RewardRange

Gymnasium's reward_range: (min(reward_schedule), max(reward_schedule)).

[PyName("reward_range")]
public (double Min, double Max) RewardRange { get; }

Property Value

(double From, double To)

S

Gymnasium's s: the agent's state, row * ncol + col; null before the first reset. Assigning one injects a state (one the map lacks makes the next step's lookup fail with KeyError).

[PyName("s")]
public long? S { get; set; }

Property Value

long?

WindowSize

Gymnasium's window_size: (min(64 * ncol, 512), min(64 * nrow, 512)) pixels.

public (int Width, int Height) WindowSize { get; }

Property Value

(int Width, int Height)

WindowSurface

Gymnasium's window_surface: the frame every render draws on, null until the first human or rgb_array render.

public Surface? WindowSurface { get; }

Property Value

Surface

Methods

Close()

Once a window surface exists, shuts the human-mode window down (pygame's display.quit()); the surface is kept.

public override void Close()

Remarks

ConvertAction(object?)

The untyped step's action conversion, with Python's order kept: the state's existence first, then the dict lookup on the action as given, so the errors name the Python counterpart of the value given (KeyError: '1' for a string).

protected override NDArray ConvertAction(object? action)

Parameters

action object

The action as given to the untyped step.

Returns

NDArray

The action as an array: the one given, or a new one the step releases.

Exceptions

AttributeError

Thrown before the first reset.

KeyError

Thrown for an action no key equals.

TypeError

Thrown for an array of one or more dimensions.

GenerateRandomMap(int, double, BigInteger?)

Gymnasium's module function generate_random_map: draws size x size tiles, frozen with probability p, from a generator seeded with seed, sets the start and goal corners, and draws again until a path joins them.

public static IReadOnlyList<string> GenerateRandomMap(int size = 8, double p = 0.8, BigInteger? seed = null)

Parameters

size int

The map's side.

p double

The probability of a frozen tile; above 1 counts as 1 (Python's min(1, p), which also turns NaN into 1).

seed BigInteger?

The seed of seeding.np_random, or null for fresh entropy.

Returns

IReadOnlyList<string>

The map's rows.

Remarks

Each attempt is one np_random.choice(["F", "H"], (size, size), p=[p, 1 - p]): the same draws as Gymnasium's, so a seed gives Gymnasium's map. Reference: gymnasium/envs/toy_text/frozen_lake.py.

Exceptions

Error

Thrown for a negative seed, with Gymnasium's message.

ValueError

Thrown when the probabilities are refused by the generator (a negative p), with NumPy's message.

IndexError

Thrown for size 0, where Python's board[0][0] fails.

IsValid(IReadOnlyList<string>, int)

Gymnasium's module function is_valid: whether a depth-first walk from the top-left tile over tiles that are not holes reaches a G inside the maxSize square.

public static bool IsValid(IReadOnlyList<string> board, int maxSize)

Parameters

board IReadOnlyList<string>

The rows of the map.

maxSize int

The square's side the walk stays within.

Returns

bool

Whether a goal is reachable.

Remarks

Exceptions

ArgumentNullException

Thrown when board is null.

Render()

Renders by the render mode: ansi returns the map as text with the agent in red; human and rgb_array draw Gymnasium's pixel art (and rgb_array returns the frame).

public override RenderResult Render()

Returns

RenderResult

ansi: the text. rgb_array: the (H, W, 3) uint8 frame, a new array the caller owns. human: nothing.

Remarks

Exceptions

AssertionError

Thrown without a render mode when the env has no spec, and for a render mode other than human, ansi and rgb_array (Something went wrong with pygame. This should never happen.), as in Gymnasium.

AttributeError

Thrown before the first reset (no s; after the map was drawn, for images).

TypeError

Thrown by ansi after an action Python's list indexing refuses (a float action).

IndexError

Thrown by ansi after an action outside the caption list, or for an injected state outside the map.

KeyError

Thrown in human mode when metadata["render_fps"] was removed.

DependencyNotInstalled

Thrown on the first human render when no viewer factory is set (DR-20).

Reset(BigInteger?, IReadOnlyDictionary<string, object?>?)

Starts an episode: reseeds the generator when a seed is given, then draws the start state from InitialStateDistrib (one draw) and clears the last action. Renders in human mode.

public override ResetResult<NDArray> Reset(BigInteger? seed = null, IReadOnlyDictionary<string, object?>? options = null)

Parameters

seed BigInteger?

The seed of a fresh Generator(PCG64(SeedSequence(seed))), or null to keep the generator.

options IReadOnlyDictionary<string, object>

Ignored, as in Gymnasium.

Returns

ResetResult<NDArray>

The state as a 0-d int64 array, and {"prob": 1} (a Python int).

Remarks

Exceptions

Error

Thrown for a negative seed, with Gymnasium's message.

DependencyNotInstalled

Thrown in human mode by the first render when no viewer factory is set (DR-20).

Step(NDArray)

Moves by the action: looks up P[s][a], draws one outcome (one float64 from the generator) and moves there. Renders in human mode.

public override StepResult<NDArray> Step(NDArray action)

Parameters

action NDArray

A number equal to 0 (left), 1 (down), 2 (right) or 3 (up): a 0-d array, or through the untyped step any .NET number.

Returns

StepResult<NDArray>

The new state (a 0-d int64 array), the reward, whether it is a goal or a hole, truncated=False, and {"prob": p}.

Remarks

Exceptions

AttributeError

Thrown before the first reset ('FrozenLakeEnv' object has no attribute 's'), before the action is looked at.

KeyError

Thrown for an action no key equals, or an injected state the map lacks, with Python's message (the value's repr).

TypeError

Thrown for an array action of one or more dimensions (unhashable type: 'numpy.ndarray').

DependencyNotInstalled

Thrown in human mode by the first render when no viewer factory is set (DR-20).