Migrating from Gym.NET 0.2 to 1.0
Documentation index ยท Getting started
Gym.NET 1.0 takes Gymnasium's API, the contract gym adopted in 0.26 and Gymnasium kept: Reset(seed) returns
(observation, info), Step returns (observation, reward, terminated, truncated, info), the render mode is fixed at
construction, and envs are made through a registry. It is a breaking release: the package ids
(Gym.NET, Gym.NET.Environments, Gym.NET.Rendering.Avalonia, Gym.NET.Rendering.WinForm) and the namespaces stay,
so most using directives keep working, but the members inside them change, and every package moves to 1.0.0 at once
(see Requirements). Gym.NET.Rendering, new, comes with Gym.NET.Environments.
Code that cannot move at once can keep the 0.2 API through Gym.Legacy.LegacyEnvAdapter (below), which is obsolete and
will be removed in 2.0.
Requirements
- .NET 8, .NET 10 or .NET 11 (0.2 targeted .NET Core 3.1 and .NET 6).
- Every Gym.NET package at 1.0.0, moved together. 0.2.0 declared its dependencies as "0.2.0 or later", so NuGet
accepts a mix of 0.2.0 and 1.0.0 packages without a warning, and the mix fails when it runs: Gym.NET.Environments
0.2.0 over Gym.NET 1.0.0 cannot load its envs (
Could not load type 'Gym.Envs.Env'), and a 0.2.0 viewer over Gym.NET.Environments 1.0.0 cannot load its window (Could not load type 'Gym.Environments.IEnvViewer'). - NumSharp 0.81.0, which the packages bring, in place of 0.2's NumSharp.Lite 0.1.12. Remove any NumSharp.Lite
reference: with both, code naming
NDArraystops compiling (CS0433, the type is in both). NumSharp's own API differs from NumSharp.Lite's: dtypes areDTypes (np.float32still reads the same), indices arelong, and random generators are NumPy'sGenerator. - Avalonia 12 for Gym.NET.Rendering.Avalonia (0.2's was built on Avalonia 0.10).
- In a build that treats warnings as errors, two suppressions, both explained, with the lines to add, under Breaking Changes in the release notes: ImageSharp 2.1.13's five advisories, and, with Gym.NET.Rendering.WinForm, NU1701 for the .NET Framework packages ScottPlot.WinForms brings.
The calls, old and new
| 0.2 | 1.0 |
|---|---|
new CartPoleEnv() |
Gymnasium.Make("CartPole-v1"), which adds Gymnasium's wrappers (TimeLimit of 500 steps, OrderEnforcing, PassiveEnvChecker); new CartPoleEnv() still builds the bare env |
new CartPoleEnv(WinFormEnvViewer.Factory) |
Gymnasium.HumanViewerFactory = WinFormEnvViewer.Factory; then Gymnasium.Make("CartPole-v1", new { render_mode = "human" }), or new CartPoleEnv(renderMode: "human", viewerFactory: WinFormEnvViewer.Factory) |
env.Seed(s); env.Reset() |
env.Reset(seed: s) |
NDArray obs = env.Reset() |
var (obs, info) = env.Reset(), or env.Reset().Observation |
Step step = env.Step(a) |
var (obs, reward, terminated, truncated, info) = env.Step(a) (a StepResult<TObs>) |
step.Done |
terminated \|\| truncated: terminated when the task ended (the pole fell), truncated when a limit cut the episode short |
step.Reward (float) |
reward (double) |
step.Information (Gym.Collections.Dict, or null) |
info (Gym.Envs.Info, never null); step.Information still compiles, obsolete, until 2.0 (DR-78). info.Get<T>(key) reads a value as T, converting NumPy scalars, and a built-in env's info also reads as a typed record (HalfCheetahInfoV5 typed = step.Info;) |
dict1.Equals(dict2) (0.2's Dict compared by value) |
info1.PyEquals(info2): Python's dict ==; an Info's Equals and == compare references (DR-92) |
env.Render("rgb_array") returning an ImageSharp Image |
build the env with render_mode = "rgb_array" (or RenderMode.RgbArray); env.Render() returns RenderResult.Rgb, whose Frame is a (height, width, 3) uint8 array; (Image)env.Render() makes an ImageSharp image of it |
env.Render("human") after each step |
build the env with render_mode = "human" (or RenderMode.Human); every Reset and Step draws the frame itself, paced at the env's frame rate |
A viewer of your own: IEnvViewer.Render(Image img), CloseEnvironment(), a factory returning Task<IEnvViewer> (IEnvironmentViewerFactoryDelegate) |
IEnvViewer.Render(NDArray frame) (the (H, W, 3) uint8 frame, the caller's), Close(), and an EnvViewerFactory that returns the viewer once its window is up; IAsyncEnvViewer adds RenderAsync/CloseAsync (DR-85). CloseEnvironment() stays, obsolete, until 2.0 |
env.CloseEnvironment() |
env.Close(), or Dispose() (using) |
Task<Step> env.StepAsync(a) |
ValueTask<StepResult<TObs>> env.StepAsync(a, cancellationToken), which runs the step on the calling thread |
env.Metadata (Dict, keys render.modes, video.frames_per_second) |
env.Metadata (Gym.Envs.Metadata, keys render_modes, render_fps) |
env.RewardRange |
removed, as in Gymnasium 1.0 |
class MyEnv : Env, class MyEnv : Env<TAction> |
class MyEnv : Env<TObs, TAct> (Creating custom environments) |
GoalEnv |
removed (Gymnasium-Robotics' concern) |
Gym.Threading (DistributedScheduler, DistributedThread, DistributedQueuedTaskScheduler, ThreadAffinityDistrubution, CancellationSourceExtensions), which 0.2's StepAsync ran on |
removed: built on Thread.Abort, which throws on .NET 5 and later. StepAsync runs the step on the calling thread; Gymnasium.MakeVec(id, n, "async") steps envs on worker threads |
IVecEnv, VecEnv, DummyVecEnv, VecEnvWrapper |
Gym.Vector: Gymnasium.MakeVec("CartPole-v1", n), SyncVectorEnv, VectorEnv<TObs, TAct>; the old types moved to Gym.Legacy, obsolete |
catch (InvalidActionError) |
compiles, with a warning: InvalidActionError is now the obsolete base of Gymnasium's InvalidAction (DR-58), which CarRacing raises and a custom env should throw. The envs that check their action assert it, as Gymnasium's do, and raise AssertionError (LunarLander, CartPole, ...), which this catch does not see |
AlreadySteppingError, NotSteppingError |
compile, with a warning: the obsolete bases of the AlreadyPendingCallError and NoAsyncCallError that AsyncVectorEnv raises (DR-58), so a 0.2 catch now catches them (0.2 itself threw neither) |
Gym.Collections.CircularQueue<T>, CircularQueueThreadSafe<T> |
removed: a Queue<T> cycled with queue.Enqueue(queue.Dequeue()), or ConcurrentQueue<T> |
Gym.Reflection.GenericCallFactory |
removed: method.MakeGenericMethod(type).Invoke(target, args) |
space.Seed(int) |
space.Seed(seed), which also takes Gymnasium's lists and dicts of seeds for composite spaces |
new Discrete(n) samples |
0-d int64 arrays, start to start + n - 1 |
new Box(low, high, shape, typeof(float)) |
new Box(low, high, shape, dtype: np.float32) |
An episode, before and after
0.2:
using var env = new CartPoleEnv(WinFormEnvViewer.Factory);
env.Seed(42);
NDArray observation = env.Reset();
while (true) {
Step step = env.Step(env.ActionSpace.Sample());
env.Render();
if (step.Done)
break;
observation = step.Observation;
}
env.CloseEnvironment();
1.0:
Gymnasium.HumanViewerFactory = WinFormEnvViewer.Factory; // once, at start-up
using Env<NDArray, NDArray> env = Gymnasium.Make<NDArray, NDArray>("CartPole-v1", new { render_mode = "human" });
env.ActionSpace.Seed(42);
NDArray observation = env.Reset(seed: 42).Observation;
while (true) {
using NDArray action = env.ActionSpace.Sample();
var (next, reward, terminated, truncated, info) = env.Step(action); // draws the frame
observation.Dispose(); // arrays belong to the caller
observation = next;
if (terminated || truncated)
break;
}
observation.Dispose();
What behaves differently, beyond the spelling:
- Episodes end on either flag.
Gymnasium.Make("CartPole-v1")truncates at 500 steps, as Gymnasium's CartPole-v1; 0.2'sCartPoleEnvran until the pole fell. - The numbers are Gymnasium's. A seed now selects Gymnasium's random stream (NumPy's
Generator(PCG64(SeedSequence(seed))), not 0.2'sRandomState(seed)), and CartPole returns float32 observations, as Gymnasium's does. The same seed gives the same trajectory asgymnasium.make("CartPole-v1")in Python, but not as 0.2 did. - Order is enforced. A
Stepbefore the firstResetraisesResetNeeded; stepping on after an episode ended warns with Gymnasium's message. - Arrays belong to the caller. NumSharp arrays live in unmanaged memory; dispose what an env returns. NumSharp's
analyzer, which comes with the packages, points at arrays that are dropped (
NDW012). - Spaces belong to their env. An env disposes the spaces it was given, so one space cannot serve two envs: assigning
a space another undisposed env owns throws. Give each env its own, a new one or
space.DeepCopy().
Rendering to an image
0.2 returned an ImageSharp Image; 1.0 returns the frame as an array, the way Gymnasium does, and converts it to an
image on request:
using Env<NDArray, NDArray> env = Gymnasium.Make<NDArray, NDArray>("CartPole-v1", RenderMode.RgbArray);
using NDArray observation = env.Reset(seed: 0).Observation;
using RenderResult result = env.Render();
NDArray frame = (NDArray)result; // (400, 600, 3) uint8, the result's own array
using Image image = (Image)result; // a new Image<Rgb24> of the frame, yours to dispose
Keeping 0.2 code running: Gym.Legacy
LegacyEnvAdapter exposes any 1.0 env through the 0.2 interface (ILegacyEnv):
#pragma warning disable CS0618 // Gym.Legacy is obsolete on purpose: it keeps 0.2 code running while it moves
using var env = new Gym.Legacy.LegacyEnvAdapter(Gymnasium.Make("CartPole-v1"));
env.Seed(42); // recorded for the next Reset, as Reset(seed: 42)
NDArray observation = env.Reset(); // the observation alone
Gym.Legacy.Step step = env.Step(1); // (Observation, Reward, Done, Information)
while (!step.Done) // Done = terminated || truncated
step = env.Step(1);
#pragma warning restore CS0618
Seed(s)records the seed for the nextReset();Reset()drops the info.Stepreportsterminated || truncatedasDoneand, when a time limit ended the episode, setsInformation["TimeLimit.truncated"], as gym 0.21 did. The reward stays adouble.Render(mode)accepts only the mode the env was built with, and returns anImageforrgb_array.- The 0.2 vector types,
IVecEnv,VecEnv,DummyVecEnvandVecEnvWrapper, are inGym.Legacytoo, overILegacyEnv.
Everything in Gym.Legacy is [Obsolete], with the replacement named in the warning, and is removed in 2.0.
The legacy Dict and its helpers
Gym.Legacy still hands out 0.2's Gym.Collections.Dict (Step.Information, ILegacyEnv.Metadata,
VecEnv.Metadata), so Dict and the fifteen types behind it stay in 1.0, [Obsolete] (DR-56), and go in 2.0 with
Gym.Legacy. Each warning names the replacement:
| 0.2 type | 1.0 replacement |
|---|---|
Gym.Collections.Dict |
Gym.Envs.Info for infos, Gym.Envs.Metadata for metadata |
Gym.Dynamic.Expando |
System.Dynamic.ExpandoObject |
Gym.Merging.Merger, IMergable, IMergable<T>, MergePrefer |
Info.Update, Python's dict.update: the new value wins, one level deep |
Gym.Collections.ExpandableList, ExpandableList<T>, IExpandableList |
List<T> |
Gym.Reflection.DefaultConstructor, DefaultConstructor<T>, DefaultConstructorNotFoundException |
Activator.CreateInstance<T>(), or a new() constraint |
Gym.Reflection.ReflectionHelper |
Type.GetConstructor(Type.EmptyTypes), Type.IsAssignableTo, System.Runtime.CompilerServices.ITuple |
DictInlineInitializationException, DictMergingException |
none needed |
Moving code from Dict to Info, member by member:
Dict |
Info |
|---|---|
new Dict("key", value, ...), Dict.Empty |
new Info { ["key"] = value, ... }, new Info() |
d["key"], ContainsKey, TryGetValue, Remove, foreach |
the same |
d.Get<T>("key") (default(T) when missing) |
info.Get<T>("key"), the same read (default(T) when missing), which also converts NumPy scalars and numbers (DR-78); or (T)info["key"]!, or info.Get("key") is T value (Python's get: null when missing) |
d.Get<T>("key", fallback) |
info.Get("key", fallback), which returns object? |
d.Update(other), Append(other), Override(other) |
info.Update(other) |
d.Merge(other, MergePrefer.New) |
info.Update(other), one level deep; nothing replaces the deep merge |
d.Clone() (shallow) |
info.Copy() (shallow) or info.DeepCopy() |
d.RemoveWhere(predicate) |
foreach (string key in info.Keys.Where(...).ToList()) info.Remove(key); |
d.AsDynamic() |
a System.Dynamic.ExpandoObject filled from the info |
Info keeps Python's insertion order through removals, which Dict (a Dictionary) does not promise.
Info also takes Dict's two reads (DR-58): info.Get<T>("key") (default(T) when missing) and info.Clone()
(the same as Copy()). Get<T> is an extension in Gym.Envs (InfoExtensions, DR-78), so it reads any string-keyed
dict and converts what Gymnasium's envs store: info.Get<double>("x_position") reads a NumPy scalar (a 0-d array),
which a plain cast refuses; an integer is range-checked, and a float reads as an integer only when it is whole. There
is no Get<T>("key", fallback): it would capture Get("key", fallback) calls, which must keep returning Python's
object.
0.2 lines that still compile
Compiling 0.2's own tests against 1.0 found the lines below broken; each now compiles with an [Obsolete] warning that
names the 1.0 spelling (DR-58), and goes in 2.0 with Gym.Legacy:
| 0.2 line | What it does in 1.0 | 1.0 spelling |
|---|---|---|
var (obs, reward, done, info) = env.Step(a); |
done is terminated \|\| truncated, as gym 0.21 reported it |
var (obs, reward, terminated, truncated, info) = env.Step(a); |
NDArray obs = env.Reset(); |
the observation; the info is dropped, and arrays in it are never disposed (Taxi's action mask) | env.Reset().Observation, or var (obs, info) = env.Reset(); |
env.CloseEnvironment(); |
Close() |
env.Close(), or using |
new Box(-5f, 5f, np.float32), new Box(low, high, dType: np.float32) |
the box dtype: builds; new Box(low, high, null) is then ambiguous |
new Box(-5, 5, dtype: np.float32) |
new CartPoleEnv(WinFormEnvViewer.Factory) |
human mode in that window, drawing every step itself (drop 0.2's per-step Render(), which would draw twice) |
new CartPoleEnv(renderMode: "human", viewerFactory: WinFormEnvViewer.Factory) |
new CartPoleEnv(NullEnvViewer.Factory) |
headless, no render mode (Render() warns once and returns nothing) |
new CartPoleEnv() |
(int)LunarLanderDiscreteActions.FireMainEngine |
2, Gymnasium's main engine (0 nothing, 1 left, 3 right); the enum is in Gym.Environments.Envs.Box2D, and in 0.2's Gym.Environments.Envs.Aether, revived for it alone (importing both is ambiguous) |
2 |
env.ContinuousMode, LunarLanderEnv.SCALE, MAIN_ENGINE_POWER, SIDE_ENGINE_POWER, INITIAL_RANDOM |
the same values, 0.2's float constants | env.Continuous, LunarLanderEnv.Scale, ... (doubles) |
Still broken: env.Seed(s) (Reset(seed: s); a Seed method would make check_env warn on every env, as Gymnasium's
does), env.RewardRange (Gymnasium 1.0 removed it), env.Render("rgb_array") returning an Image (see Rendering to an
image), NumSharp's now-explicit (float)array, and 0.2's Aether LunarLanderEnv.
LunarLander
The LunarLanderEnv of 0.2 (Gym.Environments.Envs.Aether, Aether physics) is gone, with its
LunarLanderDiscreteActions enum and the Aether.Physics2D, ImageSharp.Drawing and SixLabors.Fonts dependencies. Its
replacement is Gymnasium's LunarLander on Gym.NET's Box2D engine: Gymnasium.Make("LunarLander-v3") or
Gymnasium.Make("LunarLanderContinuous-v3"), or Gym.Environments.Envs.Box2D.LunarLanderEnv built directly.
| 0.2 | 1.0 |
|---|---|
new LunarLanderEnv(viewerFactory, random_state: rng, continuous: c, render_mode: m) |
new LunarLanderEnv(renderMode: m, continuous: c, viewerFactory: f); seed through Reset(seed), or assign env.NpRandom |
(int)LunarLanderDiscreteActions.FireMainEngine |
the action 2 (0 nothing, 1 left engine, 2 main engine, 3 right engine), as Gymnasium's |
env.ContinuousMode |
env.Continuous |
LunarLanderEnv.SCALE, INITIAL_RANDOM, ... (mutable fields) |
LunarLanderEnv.Scale, InitialRandom, ... (constants) |
| v2 observation bounds, a 0-d continuous action space | v3's bounds, Box(-1, 1, (2,), float32) |
The episodes differ from 0.2's: the physics is Box2D 2.3.0, bit for bit as Gymnasium's pybox2d, and the draws follow
Gymnasium's order, so reset(seed) gives Gymnasium's episode.