Table of Contents

Class HumanoidEnvV5

Namespace
Gym.Environments.Envs.MuJoCo
Assembly
Gym.Environments.dll

Humanoid-v5: a 3D humanoid (torso, abdomen, legs and arms on a free joint) driven by 17 joint torques in [-0.4, 0.4]. The reward pays the center of mass's forward velocity and staying healthy, and charges the squared controls and the external contact forces; a torso height outside (1, 2) ends the episode. Byte for byte what Gymnasium's Humanoid-v5 computes on MuJoCo 4.0.0 (DR-61, DR-64).

[PyName("HumanoidEnv")]
public class HumanoidEnvV5 : MujocoEnv, IEnv, IDisposable, IDynamicAttributes
Inheritance
HumanoidEnvV5
Implements
Inherited Members
Extension Methods

Remarks

Observation (float64): the positions without the root's x and y (unless asked for), the velocities, then (each unless switched off) the bodies' inertias cinert and velocities cvel and the external forces cfrc_ext, all without the world body's row, and the actuator forces qfrc_actuator without the root's 6 dofs: 348 values by default. Reward: forward_reward_weight * x_velocity + healthy_reward * is_healthy - ctrl_cost_weight * sum(ctrl**2) - clip(contact_cost_weight * sum(cfrc_ext**2), *contact_cost_range), all float64 (the control cost reads data.ctrl, so it does not depend on the action's dtype). Info: x_position, y_position, tendon_length, tendon_velocity, distance_from_origin, x_velocity, y_velocity, reward_survive, reward_forward, reward_ctrl, reward_contact; the reset's info holds the first five.

Not thread-safe. Reference: gymnasium/envs/mujoco/humanoid_v5.py.

Its info reads typed (DR-91): HumanoidInfoV5 info = step.Info; (HumanoidInfoV5), with the dict's own values still behind the indexer.

Constructors

HumanoidEnvV5(HumanoidEnvV5Options)

Builds the instance from its options record: the properties set, the constructor's defaults for the rest.

public HumanoidEnvV5(HumanoidEnvV5Options options)

Parameters

options HumanoidEnvV5Options

The options.

Exceptions

ArgumentNullException

Thrown when options is null.

HumanoidEnvV5(RenderMode, string, int, IReadOnlyDictionary<string, object?>?, double, double, double, (double Low, double High)?, double, bool, (double Low, double High)?, double, bool, bool, bool, bool, bool, int, int, int?, string?, int, IReadOnlyDictionary<int, bool>?, EnvViewerFactory?)

Builds the env with the render mode given as the RenderMode enum (DR-84, DR-93), as HumanoidEnvV5(string, int, IReadOnlyDictionary<string, object?>?, double, double, double, (double Low, double High)?, double, bool, (double Low, double High)?, double, bool, bool, bool, bool, bool, string?, int, int, int?, string?, int, IReadOnlyDictionary<int, bool>?, EnvViewerFactory?) builds it from the mode's string: new HumanoidEnvV5(RenderMode.RgbArray) builds what new HumanoidEnvV5(renderMode: "rgb_array") builds, the member reaching the env as Gymnasium's string (Name(RenderMode)), which its RenderMode property then holds.

public HumanoidEnvV5(RenderMode renderMode, string xmlFile = "humanoid.xml", int frameSkip = 5, IReadOnlyDictionary<string, object?>? defaultCameraConfig = null, double forwardRewardWeight = 1.25, double ctrlCostWeight = 0.1, double contactCostWeight = 5E-07, (double Low, double High)? contactCostRange = null, double healthyReward = 5, bool terminateWhenUnhealthy = true, (double Low, double High)? healthyZRange = null, double resetNoiseScale = 0.01, bool excludeCurrentPositionsFromObservation = true, bool includeCinertInObservation = true, bool includeCvelInObservation = true, bool includeQfrcActuatorInObservation = true, bool includeCfrcExtInObservation = true, int width = 480, int height = 480, int? cameraId = null, string? cameraName = null, int maxGeom = 1000, IReadOnlyDictionary<int, bool>? visualOptions = null, EnvViewerFactory? viewerFactory = null)

Parameters

renderMode RenderMode

The render mode, which the env must list in its metadata["render_modes"] as for the string; it behaves exactly as its Gymnasium string does in the other constructor.

xmlFile string

The model: a bundled file name (xml_file, DR-63).

frameSkip int

Physics steps per env step (frame_skip).

defaultCameraConfig IReadOnlyDictionary<string, object>

The free camera's settings (default_camera_config); null is Gymnasium's default (tracking body 1 from 4 m, looking at (0, 0, 2), 20° from above), and an empty dictionary leaves MuJoCo's camera as it is.

forwardRewardWeight double

The weight of the forward velocity (forward_reward_weight).

ctrlCostWeight double

The weight of the squared controls (ctrl_cost_weight).

contactCostWeight double

The weight of the squared external forces (contact_cost_weight).

contactCostRange (double hx, double hy)?

The range the contact cost is clipped to (contact_cost_range); null is (-inf, 10.0).

healthyReward double

The reward for each healthy step (healthy_reward).

terminateWhenUnhealthy bool

Whether an unhealthy state ends the episode (terminate_when_unhealthy).

healthyZRange (double hx, double hy)?

The exclusive range of a healthy torso height (healthy_z_range); null is (1.0, 2.0).

resetNoiseScale double

The scale of the reset's noise (reset_noise_scale).

excludeCurrentPositionsFromObservation bool

Whether the root's x and y leave the observation (exclude_current_positions_from_observation).

includeCinertInObservation bool

Whether the bodies' inertias join the observation (include_cinert_in_observation).

includeCvelInObservation bool

Whether the bodies' velocities join the observation (include_cvel_in_observation).

includeQfrcActuatorInObservation bool

Whether the actuator forces join the observation (include_qfrc_actuator_in_observation).

includeCfrcExtInObservation bool

Whether the external forces join the observation (include_cfrc_ext_in_observation).

width int

The rendered frame's width (width).

height int

The rendered frame's height (height).

cameraId int?

The camera to render from (camera_id).

cameraName string

The camera to render from by name (camera_name).

maxGeom int

The most geoms a frame draws (max_geom).

visualOptions IReadOnlyDictionary<int, bool>

MuJoCo visual flags to switch (visual_options).

viewerFactory EnvViewerFactory

Gym.NET's viewer_factory (DR-20) for human mode.

Remarks

The mode is required and comes first, so new HumanoidEnvV5() keeps meaning the other constructor, without a mode; leave the mode out there for no rendering. Every other parameter means what it means there.

Exceptions

ArgumentOutOfRangeException

Thrown for a value RenderMode does not declare (a cast integer such as (RenderMode)99).

NotSupportedException

xmlFile is not a bundled model.

OverflowError

A frame skip of 0 (Python's OverflowError deriving the frame rate).

ValueError

Both a camera id and a camera name.

HumanoidEnvV5(string, int, IReadOnlyDictionary<string, object?>?, double, double, double, (double Low, double High)?, double, bool, (double Low, double High)?, double, bool, bool, bool, bool, bool, string?, int, int, int?, string?, int, IReadOnlyDictionary<int, bool>?, EnvViewerFactory?)

Creates the env (Gymnasium's HumanoidEnv.init): the model, its spaces and the frame rate the time step gives.

public HumanoidEnvV5(string xmlFile = "humanoid.xml", int frameSkip = 5, IReadOnlyDictionary<string, object?>? defaultCameraConfig = null, double forwardRewardWeight = 1.25, double ctrlCostWeight = 0.1, double contactCostWeight = 5E-07, (double Low, double High)? contactCostRange = null, double healthyReward = 5, bool terminateWhenUnhealthy = true, (double Low, double High)? healthyZRange = null, double resetNoiseScale = 0.01, bool excludeCurrentPositionsFromObservation = true, bool includeCinertInObservation = true, bool includeCvelInObservation = true, bool includeQfrcActuatorInObservation = true, bool includeCfrcExtInObservation = true, string? renderMode = null, int width = 480, int height = 480, int? cameraId = null, string? cameraName = null, int maxGeom = 1000, IReadOnlyDictionary<int, bool>? visualOptions = null, EnvViewerFactory? viewerFactory = null)

Parameters

xmlFile string

The model: a bundled file name (xml_file, DR-63).

frameSkip int

Physics steps per env step (frame_skip).

defaultCameraConfig IReadOnlyDictionary<string, object>

The free camera's settings (default_camera_config); null is Gymnasium's default (tracking body 1 from 4 m, looking at (0, 0, 2), 20° from above), and an empty dictionary leaves MuJoCo's camera as it is.

forwardRewardWeight double

The weight of the forward velocity (forward_reward_weight).

ctrlCostWeight double

The weight of the squared controls (ctrl_cost_weight).

contactCostWeight double

The weight of the squared external forces (contact_cost_weight).

contactCostRange (double hx, double hy)?

The range the contact cost is clipped to (contact_cost_range); null is (-inf, 10.0).

healthyReward double

The reward for each healthy step (healthy_reward).

terminateWhenUnhealthy bool

Whether an unhealthy state ends the episode (terminate_when_unhealthy).

healthyZRange (double hx, double hy)?

The exclusive range of a healthy torso height (healthy_z_range); null is (1.0, 2.0).

resetNoiseScale double

The scale of the reset's noise (reset_noise_scale).

excludeCurrentPositionsFromObservation bool

Whether the root's x and y leave the observation (exclude_current_positions_from_observation).

includeCinertInObservation bool

Whether the bodies' inertias join the observation (include_cinert_in_observation).

includeCvelInObservation bool

Whether the bodies' velocities join the observation (include_cvel_in_observation).

includeQfrcActuatorInObservation bool

Whether the actuator forces join the observation (include_qfrc_actuator_in_observation).

includeCfrcExtInObservation bool

Whether the external forces join the observation (include_cfrc_ext_in_observation).

renderMode string

Gymnasium's render_mode: human, rgb_array, depth_array, rgbd_tuple or null.

width int

The rendered frame's width (width).

height int

The rendered frame's height (height).

cameraId int?

The camera to render from (camera_id).

cameraName string

The camera to render from by name (camera_name).

maxGeom int

The most geoms a frame draws (max_geom).

visualOptions IReadOnlyDictionary<int, bool>

MuJoCo visual flags to switch (visual_options).

viewerFactory EnvViewerFactory

Gym.NET's viewer_factory (DR-20) for human mode.

Exceptions

NotSupportedException

xmlFile is not a bundled model.

OverflowError

A frame skip of 0 (Python's OverflowError deriving the frame rate).

ValueError

Both a camera id and a camera name.

Properties

ClassMetadata

Gymnasium's class attribute metadata: the four render modes, the frame rate set per instance. A new dict on every read.

public static Metadata ClassMetadata { get; }

Property Value

Metadata

ContactCost

The contact cost now (contact_cost): contact_cost_weight * sum(cfrc_ext**2) clipped to ContactCostRange.

[PyName("contact_cost")]
public double ContactCost { get; }

Property Value

double

ContactCostRange

The range the contact cost is clipped to (_contact_cost_range).

[PyName("_contact_cost_range")]
public (double Low, double High) ContactCostRange { get; set; }

Property Value

(double From, double To)

ContactCostWeight

The weight of the squared external forces (_contact_cost_weight), read every step.

[PyName("_contact_cost_weight")]
public double ContactCostWeight { get; set; }

Property Value

double

CtrlCostWeight

The weight of the squared controls (_ctrl_cost_weight), read every step.

[PyName("_ctrl_cost_weight")]
public double CtrlCostWeight { get; set; }

Property Value

double

ExcludeCurrentPositionsFromObservation

Whether the root's x and y leave the observation (_exclude_current_positions_from_observation). Footgun: this and the four Include... flags change the observation's length when changed after construction, but not the observation space, as in Gymnasium.

[PyName("_exclude_current_positions_from_observation")]
public bool ExcludeCurrentPositionsFromObservation { get; set; }

Property Value

bool

ForwardRewardWeight

The weight of the forward velocity (_forward_reward_weight), read every step.

[PyName("_forward_reward_weight")]
public double ForwardRewardWeight { get; set; }

Property Value

double

HealthyReward

The healthy reward now (healthy_reward): is_healthy * _healthy_reward, an np.float64 there.

[PyName("healthy_reward")]
public double HealthyReward { get; }

Property Value

double

HealthyRewardWeight

The reward for each healthy step (_healthy_reward), read every step.

[PyName("_healthy_reward")]
public double HealthyRewardWeight { get; set; }

Property Value

double

HealthyZRange

The exclusive range of a healthy torso height (_healthy_z_range).

[PyName("_healthy_z_range")]
public (double Low, double High) HealthyZRange { get; set; }

Property Value

(double From, double To)

IncludeCfrcExtInObservation

Whether the external forces join the observation (_include_cfrc_ext_in_observation).

[PyName("_include_cfrc_ext_in_observation")]
public bool IncludeCfrcExtInObservation { get; set; }

Property Value

bool

IncludeCinertInObservation

Whether the bodies' inertias join the observation (_include_cinert_in_observation).

[PyName("_include_cinert_in_observation")]
public bool IncludeCinertInObservation { get; set; }

Property Value

bool

IncludeCvelInObservation

Whether the bodies' velocities join the observation (_include_cvel_in_observation).

[PyName("_include_cvel_in_observation")]
public bool IncludeCvelInObservation { get; set; }

Property Value

bool

IncludeQfrcActuatorInObservation

Whether the actuator forces join the observation (_include_qfrc_actuator_in_observation).

[PyName("_include_qfrc_actuator_in_observation")]
public bool IncludeQfrcActuatorInObservation { get; set; }

Property Value

bool

IsHealthy

Whether the humanoid is healthy now (is_healthy): the torso's height inside the exclusive z range.

[PyName("is_healthy")]
public bool IsHealthy { get; }

Property Value

bool

ObservationStructure

How the observation is laid out (observation_structure): skipped_qpos, qpos, qvel, cinert, cvel, qfrc_actuator, cfrc_ext, and 0 for ten_length and ten_velocity (reported in the info, not observed).

public Info ObservationStructure { get; }

Property Value

Info

ResetNoiseScale

The scale of the reset's noise (_reset_noise_scale), read every reset.

[PyName("_reset_noise_scale")]
public double ResetNoiseScale { get; set; }

Property Value

double

TerminateWhenUnhealthy

Whether an unhealthy state ends the episode (_terminate_when_unhealthy), read every step.

[PyName("_terminate_when_unhealthy")]
public bool TerminateWhenUnhealthy { get; set; }

Property Value

bool

Methods

ControlCost(NDArray?)

The control cost (control_cost): ctrl_cost_weight * sum(ctrl**2) of the simulation's controls, float64; action is taken but not read, as in Gymnasium.

[PyName("control_cost")]
public double ControlCost(NDArray? action)

Parameters

action NDArray

The action; ignored.

Returns

double

The cost.

GetResetInfo()

The reset's info (_get_reset_info): x_position, y_position, copies of tendon_length and tendon_velocity, and distance_from_origin.

protected override Info GetResetInfo()

Returns

Info

The info.

MassCenter(MjModel, MjData)

The center of mass's x and y (mass_center): einsum("b,bj->j", body_mass, xipos) / body_mass.sum(), from the bodies' inertial frames of the last forward pass. A new (2,) float64 array the caller owns.

[PyName("mass_center")]
public static NDArray MassCenter(MjModel model, MjData data)

Parameters

model MjModel

The model.

data MjData

The simulation.

Returns

NDArray

The (2,) array.

Remarks

Exceptions

ArgumentNullException

A null model or data.

ResetModel()

The episode's start (reset_model): positions init_qpos + uniform(-scale, scale, nq), then velocities init_qvel + uniform(-scale, scale, nv), drawn in that order.

protected override NDArray ResetModel()

Returns

NDArray

The first observation.

Step(NDArray)

Applies action for FrameSkip physics steps and reports the transition (step).

public override StepResult<NDArray> Step(NDArray action)

Parameters

action NDArray

The 17 controls, any real dtype (stored as float64).

Returns

StepResult<NDArray>

The observation, the float64 reward, whether an unhealthy state ended the episode, truncated=False and the info.

Exceptions

ArgumentNullException

action is null.

ValueError

The action is not of shape (17,), with Gymnasium's message.