Table of Contents

Class ReacherEnvV5

Namespace
Gym.Environments.Envs.MuJoCo
Assembly
Gym.Environments.dll

Reacher-v5: a two-joint planar arm that reaches for a random target, driven by 2 joint torques in [-1, 1]. The reward charges the fingertip's distance to the target and the squared torques; the episode never ends on its own (make's 50-step limit truncates it). Byte for byte what Gymnasium's Reacher-v5 computes on MuJoCo 4.0.0 (DR-61, DR-64).

[PyName("ReacherEnv")]
public class ReacherEnvV5 : MujocoEnv, IEnv, IDisposable, IDynamicAttributes
Inheritance
ReacherEnvV5
Implements
Inherited Members
Extension Methods

Remarks

Observation (float64, 10 values): the cosines and sines of the two arm angles, the target's two coordinates, the arm's two angular velocities, and the fingertip's offset from the target in x and y. Reward: -|fingertip - target| * reward_dist_weight - sum(action**2) * reward_control_weight, measured after the step, the control term in the action's dtype. Info: reward_dist, reward_ctrl; the reset's info is empty. The reset draws the target uniformly in the disc of radius 0.2 by rejection (Goal).

Not thread-safe. Reference: gymnasium/envs/mujoco/reacher_v5.py.

Its info reads typed (DR-91): ReacherInfoV5 info = step.Info; (ReacherInfoV5), with the dict's own values still behind the indexer.

Constructors

ReacherEnvV5(ReacherEnvV5Options)

Builds the instance from its options record: the properties set, the constructor's defaults for the rest.

public ReacherEnvV5(ReacherEnvV5Options options)

Parameters

options ReacherEnvV5Options

The options.

Exceptions

ArgumentNullException

Thrown when options is null.

ReacherEnvV5(RenderMode, string, int, IReadOnlyDictionary<string, object?>?, double, double?, int, int, int?, string?, int, IReadOnlyDictionary<int, bool>?, EnvViewerFactory?)

Builds the env with the render mode given as the RenderMode enum (DR-84, DR-93), as ReacherEnvV5(string, int, IReadOnlyDictionary<string, object?>?, double, double?, string?, int, int, int?, string?, int, IReadOnlyDictionary<int, bool>?, EnvViewerFactory?) builds it from the mode's string: new ReacherEnvV5(RenderMode.RgbArray) builds what new ReacherEnvV5(renderMode: "rgb_array") builds, the member reaching the env as Gymnasium's string (Name(RenderMode)), which its RenderMode property then holds.

public ReacherEnvV5(RenderMode renderMode, string xmlFile = "reacher.xml", int frameSkip = 2, IReadOnlyDictionary<string, object?>? defaultCameraConfig = null, double rewardDistWeight = 1, double? rewardControlWeight = null, int width = 480, int height = 480, int? cameraId = null, string? cameraName = null, int maxGeom = 1000, IReadOnlyDictionary<int, bool>? visualOptions = null, EnvViewerFactory? viewerFactory = null)

Parameters

renderMode RenderMode

The render mode, which the env must list in its metadata["render_modes"] as for the string; it behaves exactly as its Gymnasium string does in the other constructor.

xmlFile string

The model: a bundled file name (xml_file, DR-63).

frameSkip int

Physics steps per env step (frame_skip).

defaultCameraConfig IReadOnlyDictionary<string, object>

The free camera's settings (default_camera_config); null is Gymnasium's default (trackbodyid 0), and an empty dictionary leaves MuJoCo's camera as it is.

rewardDistWeight double

The weight of the fingertip's distance to the target (reward_dist_weight; Gymnasium's default is the int 1).

rewardControlWeight double?

The weight of the squared torques (reward_control_weight); null is Gymnasium's default, the Python int 1, which keeps an integer action's cost an np.int64 (a given weight is a Python float).

width int

The rendered frame's width (width).

height int

The rendered frame's height (height).

cameraId int?

The camera to render from (camera_id).

cameraName string

The camera to render from by name (camera_name).

maxGeom int

The most geoms a frame draws (max_geom).

visualOptions IReadOnlyDictionary<int, bool>

MuJoCo visual flags to switch (visual_options).

viewerFactory EnvViewerFactory

Gym.NET's viewer_factory (DR-20) for human mode.

Remarks

The mode is required and comes first, so new ReacherEnvV5() keeps meaning the other constructor, without a mode; leave the mode out there for no rendering. Every other parameter means what it means there.

Exceptions

ArgumentOutOfRangeException

Thrown for a value RenderMode does not declare (a cast integer such as (RenderMode)99).

NotSupportedException

xmlFile is not a bundled model.

OverflowError

A frame skip of 0 (Python's OverflowError deriving the frame rate).

ValueError

Both a camera id and a camera name.

ReacherEnvV5(string, int, IReadOnlyDictionary<string, object?>?, double, double?, string?, int, int, int?, string?, int, IReadOnlyDictionary<int, bool>?, EnvViewerFactory?)

Creates the env (Gymnasium's ReacherEnv.init): the model, its spaces and the frame rate the time step gives.

public ReacherEnvV5(string xmlFile = "reacher.xml", int frameSkip = 2, IReadOnlyDictionary<string, object?>? defaultCameraConfig = null, double rewardDistWeight = 1, double? rewardControlWeight = null, string? renderMode = null, int width = 480, int height = 480, int? cameraId = null, string? cameraName = null, int maxGeom = 1000, IReadOnlyDictionary<int, bool>? visualOptions = null, EnvViewerFactory? viewerFactory = null)

Parameters

xmlFile string

The model: a bundled file name (xml_file, DR-63).

frameSkip int

Physics steps per env step (frame_skip).

defaultCameraConfig IReadOnlyDictionary<string, object>

The free camera's settings (default_camera_config); null is Gymnasium's default (trackbodyid 0), and an empty dictionary leaves MuJoCo's camera as it is.

rewardDistWeight double

The weight of the fingertip's distance to the target (reward_dist_weight; Gymnasium's default is the int 1).

rewardControlWeight double?

The weight of the squared torques (reward_control_weight); null is Gymnasium's default, the Python int 1, which keeps an integer action's cost an np.int64 (a given weight is a Python float).

renderMode string

Gymnasium's render_mode: human, rgb_array, depth_array, rgbd_tuple or null.

width int

The rendered frame's width (width).

height int

The rendered frame's height (height).

cameraId int?

The camera to render from (camera_id).

cameraName string

The camera to render from by name (camera_name).

maxGeom int

The most geoms a frame draws (max_geom).

visualOptions IReadOnlyDictionary<int, bool>

MuJoCo visual flags to switch (visual_options).

viewerFactory EnvViewerFactory

Gym.NET's viewer_factory (DR-20) for human mode.

Exceptions

NotSupportedException

xmlFile is not a bundled model.

OverflowError

A frame skip of 0 (Python's OverflowError deriving the frame rate).

ValueError

Both a camera id and a camera name.

Properties

ClassMetadata

Gymnasium's class attribute metadata: the four render modes, the frame rate set per instance. A new dict on every read.

public static Metadata ClassMetadata { get; }

Property Value

Metadata

Goal

The target the last reset drew (goal), float64 (2,), or null before the first reset. Owned by the env, replaced by each reset: do not dispose it.

[NDBorrowed]
public NDArray? Goal { get; }

Property Value

NDArray

RewardControlWeight

The weight of the squared torques (_reward_control_weight), read every step. Assigning one makes it a Python float, as assigning 1.0 in Python would: an integer action's cost then becomes an np.float64.

[PyName("_reward_control_weight")]
public double RewardControlWeight { get; set; }

Property Value

double

RewardDistWeight

The weight of the fingertip's distance to the target (_reward_dist_weight), read every step.

[PyName("_reward_dist_weight")]
public double RewardDistWeight { get; set; }

Property Value

double

Methods

Dispose(bool)

Closes, then releases the last target array and everything MujocoEnv holds.

protected override void Dispose(bool disposing)

Parameters

disposing bool

true from Dispose().

ResetModel()

The episode's start (reset_model): positions uniform(-0.1, 0.1, nq) + init_qpos; the target drawn as uniform(-0.2, 0.2, 2) until its norm is below 0.2, into the last two positions; velocities init_qvel + uniform(-0.005, 0.005, nv) with the target's zeroed.

protected override NDArray ResetModel()

Returns

NDArray

The first observation.

Step(NDArray)

Applies action for FrameSkip physics steps and reports the transition (step).

public override StepResult<NDArray> Step(NDArray action)

Parameters

action NDArray

The 2 torques, float32 (the action space's samples) or any real dtype.

Returns

StepResult<NDArray>

The observation, the float64 reward, terminated=False, truncated=False and the info.

Exceptions

ArgumentNullException

action is null.

ValueError

The action is not of shape (2,), with Gymnasium's message.

NotSupportedException

An action of a dtype whose cost NumPy keeps in that dtype (float16, complex).

KeyError

The model lacks the body fingertip or target (MuJoCo's message).