Table of Contents

Class HopperEnvV4

Namespace
Gym.Environments.Envs.MuJoCo
Assembly
Gym.Environments.dll

Hopper-v4: the one-legged hopper of HopperEnvV5 with v4's fixed model and frame skip, an info of x_position and x_velocity only, and v4's healthy reward, paid on every step while terminate_when_unhealthy is set. Byte for byte what Gymnasium's Hopper-v4 computes on MuJoCo 4.0.0 (DR-61, DR-64).

[PyName("HopperEnv")]
public class HopperEnvV4 : MujocoEnv, IEnv, IDisposable, IDynamicAttributes
Inheritance
HopperEnvV4
Implements
Inherited Members
Extension Methods

Remarks

Reward: forward_reward_weight * x_velocity + float(is_healthy or terminate_when_unhealthy) * healthy_reward - ctrl_cost_weight * sum(action**2). The observation space is 11 (12 with the root's x) float64 values. The reset's info is empty; make warns that the id is out of date.

Not thread-safe. Reference: gymnasium/envs/mujoco/hopper_v4.py.

Its info reads typed (DR-91): HopperInfoV4 info = step.Info; (HopperInfoV4), with the dict's own values still behind the indexer.

Constructors

HopperEnvV4(HopperEnvV4Options)

Builds the instance from its options record: the properties set, the constructor's defaults for the rest.

public HopperEnvV4(HopperEnvV4Options options)

Parameters

options HopperEnvV4Options

The options.

Exceptions

ArgumentNullException

Thrown when options is null.

HopperEnvV4(RenderMode, double, double, double, bool, (double Low, double High)?, (double Low, double High)?, (double Low, double High)?, double, bool, int, int, int?, string?, int, IReadOnlyDictionary<int, bool>?, EnvViewerFactory?)

Builds the env with the render mode given as the RenderMode enum (DR-84, DR-93), as HopperEnvV4(double, double, double, bool, (double Low, double High)?, (double Low, double High)?, (double Low, double High)?, double, bool, string?, int, int, int?, string?, int, IReadOnlyDictionary<int, bool>?, EnvViewerFactory?) builds it from the mode's string: new HopperEnvV4(RenderMode.RgbArray) builds what new HopperEnvV4(renderMode: "rgb_array") builds, the member reaching the env as Gymnasium's string (Name(RenderMode)), which its RenderMode property then holds.

public HopperEnvV4(RenderMode renderMode, double forwardRewardWeight = 1, double ctrlCostWeight = 0.001, double healthyReward = 1, bool terminateWhenUnhealthy = true, (double Low, double High)? healthyStateRange = null, (double Low, double High)? healthyZRange = null, (double Low, double High)? healthyAngleRange = null, double resetNoiseScale = 0.005, bool excludeCurrentPositionsFromObservation = true, int width = 480, int height = 480, int? cameraId = null, string? cameraName = null, int maxGeom = 1000, IReadOnlyDictionary<int, bool>? visualOptions = null, EnvViewerFactory? viewerFactory = null)

Parameters

renderMode RenderMode

The render mode, which the env must list in its metadata["render_modes"] as for the string; it behaves exactly as its Gymnasium string does in the other constructor.

forwardRewardWeight double

The weight of the forward velocity (forward_reward_weight).

ctrlCostWeight double

The weight of the squared torques (ctrl_cost_weight).

healthyReward double

The healthy reward per step (healthy_reward).

terminateWhenUnhealthy bool

Whether an unhealthy state ends the episode (terminate_when_unhealthy).

healthyStateRange (double hx, double hy)?

The exclusive range of every coordinate but the root's x and z (healthy_state_range); null is (-100, 100).

healthyZRange (double hx, double hy)?

The exclusive range of a healthy height (healthy_z_range); null is (0.7, inf).

healthyAngleRange (double hx, double hy)?

The exclusive range of a healthy torso angle (healthy_angle_range); null is (-0.2, 0.2).

resetNoiseScale double

The scale of the reset's noise (reset_noise_scale).

excludeCurrentPositionsFromObservation bool

Whether the root's x leaves the observation (exclude_current_positions_from_observation).

width int

The rendered frame's width (width).

height int

The rendered frame's height (height).

cameraId int?

The camera to render from (camera_id).

cameraName string

The camera to render from by name (camera_name).

maxGeom int

The most geoms a frame draws (max_geom).

visualOptions IReadOnlyDictionary<int, bool>

MuJoCo visual flags to switch (visual_options).

viewerFactory EnvViewerFactory

Gym.NET's viewer_factory (DR-20) for human mode.

Remarks

The mode is required and comes first, so new HopperEnvV4() keeps meaning the other constructor, without a mode; leave the mode out there for no rendering. Every other parameter means what it means there.

Exceptions

ArgumentOutOfRangeException

Thrown for a value RenderMode does not declare (a cast integer such as (RenderMode)99).

ValueError

Both a camera id and a camera name.

HopperEnvV4(double, double, double, bool, (double Low, double High)?, (double Low, double High)?, (double Low, double High)?, double, bool, string?, int, int, int?, string?, int, IReadOnlyDictionary<int, bool>?, EnvViewerFactory?)

Creates the env (Gymnasium's HopperEnv.init): hopper.xml with a frame skip of 4.

public HopperEnvV4(double forwardRewardWeight = 1, double ctrlCostWeight = 0.001, double healthyReward = 1, bool terminateWhenUnhealthy = true, (double Low, double High)? healthyStateRange = null, (double Low, double High)? healthyZRange = null, (double Low, double High)? healthyAngleRange = null, double resetNoiseScale = 0.005, bool excludeCurrentPositionsFromObservation = true, string? renderMode = null, int width = 480, int height = 480, int? cameraId = null, string? cameraName = null, int maxGeom = 1000, IReadOnlyDictionary<int, bool>? visualOptions = null, EnvViewerFactory? viewerFactory = null)

Parameters

forwardRewardWeight double

The weight of the forward velocity (forward_reward_weight).

ctrlCostWeight double

The weight of the squared torques (ctrl_cost_weight).

healthyReward double

The healthy reward per step (healthy_reward).

terminateWhenUnhealthy bool

Whether an unhealthy state ends the episode (terminate_when_unhealthy).

healthyStateRange (double hx, double hy)?

The exclusive range of every coordinate but the root's x and z (healthy_state_range); null is (-100, 100).

healthyZRange (double hx, double hy)?

The exclusive range of a healthy height (healthy_z_range); null is (0.7, inf).

healthyAngleRange (double hx, double hy)?

The exclusive range of a healthy torso angle (healthy_angle_range); null is (-0.2, 0.2).

resetNoiseScale double

The scale of the reset's noise (reset_noise_scale).

excludeCurrentPositionsFromObservation bool

Whether the root's x leaves the observation (exclude_current_positions_from_observation).

renderMode string

Gymnasium's render_mode: human, rgb_array, depth_array, rgbd_tuple or null.

width int

The rendered frame's width (width).

height int

The rendered frame's height (height).

cameraId int?

The camera to render from (camera_id).

cameraName string

The camera to render from by name (camera_name).

maxGeom int

The most geoms a frame draws (max_geom).

visualOptions IReadOnlyDictionary<int, bool>

MuJoCo visual flags to switch (visual_options).

viewerFactory EnvViewerFactory

Gym.NET's viewer_factory (DR-20) for human mode.

Exceptions

ValueError

Both a camera id and a camera name.

Properties

ClassMetadata

Gymnasium's class attribute metadata: the four render modes at 125 frames per second. A new dict on every read.

public static Metadata ClassMetadata { get; }

Property Value

Metadata

CtrlCostWeight

The weight of the squared torques (_ctrl_cost_weight), read every step.

[PyName("_ctrl_cost_weight")]
public double CtrlCostWeight { get; set; }

Property Value

double

ExcludeCurrentPositionsFromObservation

Whether the root's x leaves the observation (_exclude_current_positions_from_observation). Footgun: changing it after construction changes the observation's length but not the observation space, as in Gymnasium.

[PyName("_exclude_current_positions_from_observation")]
public bool ExcludeCurrentPositionsFromObservation { get; set; }

Property Value

bool

ForwardRewardWeight

The weight of the forward velocity (_forward_reward_weight), read every step.

[PyName("_forward_reward_weight")]
public double ForwardRewardWeight { get; set; }

Property Value

double

HealthyAngleRange

The exclusive range of a healthy torso angle (_healthy_angle_range).

[PyName("_healthy_angle_range")]
public (double Low, double High) HealthyAngleRange { get; set; }

Property Value

(double From, double To)

HealthyReward

The healthy reward now (healthy_reward): float(is_healthy or terminate_when_unhealthy) * _healthy_reward, so paid on every step while unhealthy states end the episode.

[PyName("healthy_reward")]
public double HealthyReward { get; }

Property Value

double

HealthyRewardWeight

The healthy reward per step (_healthy_reward), read every step.

[PyName("_healthy_reward")]
public double HealthyRewardWeight { get; set; }

Property Value

double

HealthyStateRange

The exclusive range of every coordinate but the root's x and z (_healthy_state_range).

[PyName("_healthy_state_range")]
public (double Low, double High) HealthyStateRange { get; set; }

Property Value

(double From, double To)

HealthyZRange

The exclusive range of a healthy height (_healthy_z_range).

[PyName("_healthy_z_range")]
public (double Low, double High) HealthyZRange { get; set; }

Property Value

(double From, double To)

IsHealthy

Whether the hopper is healthy now (is_healthy): every coordinate but the root's x and z inside the state range, the height and the torso angle inside theirs, all exclusive (a NaN is unhealthy).

[PyName("is_healthy")]
public bool IsHealthy { get; }

Property Value

bool

ResetNoiseScale

The scale of the reset's noise (_reset_noise_scale), read every reset.

[PyName("_reset_noise_scale")]
public double ResetNoiseScale { get; set; }

Property Value

double

TerminateWhenUnhealthy

Whether an unhealthy state ends the episode (_terminate_when_unhealthy), read every step.

[PyName("_terminate_when_unhealthy")]
public bool TerminateWhenUnhealthy { get; set; }

Property Value

bool

Terminated

Whether the episode ended (terminated): unhealthy while terminate_when_unhealthy is set.

[PyName("terminated")]
public bool Terminated { get; }

Property Value

bool

Methods

ControlCost(NDArray)

The control cost of action (control_cost): ctrl_cost_weight * sum(action**2), computed in the action's dtype and returned widened exactly.

[PyName("control_cost")]
public double ControlCost(NDArray action)

Parameters

action NDArray

The action.

Returns

double

The cost.

Exceptions

ArgumentNullException

action is null.

NotSupportedException

An action of a dtype whose cost NumPy keeps in that dtype (float16, complex).

ResetModel()

The episode's start (reset_model): positions init_qpos + uniform(-scale, scale, nq), then velocities init_qvel + uniform(-scale, scale, nv), drawn in that order.

protected override NDArray ResetModel()

Returns

NDArray

The first observation.

Step(NDArray)

Applies action for 4 physics steps and reports the transition (step).

public override StepResult<NDArray> Step(NDArray action)

Parameters

action NDArray

The 3 torques, float32 (the action space's samples) or any real dtype.

Returns

StepResult<NDArray>

The observation, the float64 reward, Terminated, truncated=False and the info.

Exceptions

ArgumentNullException

action is null.

ValueError

The action is not of shape (3,), with Gymnasium's message.

NotSupportedException

An action of a dtype whose cost NumPy keeps in that dtype (float16, complex).