Class HumanoidEnvV5
- Namespace
- Gym.Environments.Envs.MuJoCo
- Assembly
- Gym.Environments.dll
Humanoid-v5: a 3D humanoid (torso, abdomen, legs and arms on a free joint) driven by 17 joint torques in [-0.4, 0.4]. The reward pays the center of mass's forward velocity and staying healthy, and charges the squared controls and the external contact forces; a torso height outside (1, 2) ends the episode. Byte for byte what Gymnasium's Humanoid-v5 computes on MuJoCo 4.0.0 (DR-61, DR-64).
[PyName("HumanoidEnv")]
public class HumanoidEnvV5 : MujocoEnv, IEnv, IDisposable, IDynamicAttributes
- Inheritance
-
HumanoidEnvV5
- Implements
- Inherited Members
- Extension Methods
Remarks
Observation (float64): the positions without the root's x and y (unless asked for), the velocities, then (each
unless switched off) the bodies' inertias cinert and velocities cvel and the external forces
cfrc_ext, all without the world body's row, and the actuator forces qfrc_actuator without the
root's 6 dofs: 348 values by default. Reward: forward_reward_weight * x_velocity + healthy_reward *
is_healthy - ctrl_cost_weight * sum(ctrl**2) - clip(contact_cost_weight * sum(cfrc_ext**2), *contact_cost_range),
all float64 (the control cost reads data.ctrl, so it does not depend on the action's dtype). Info:
x_position, y_position, tendon_length, tendon_velocity,
distance_from_origin, x_velocity, y_velocity, reward_survive,
reward_forward, reward_ctrl, reward_contact; the reset's info holds the first five.
Not thread-safe. Reference: gymnasium/envs/mujoco/humanoid_v5.py.
Its info reads typed (DR-91): HumanoidInfoV5 info = step.Info; (HumanoidInfoV5), with the dict's own values still behind the indexer.
Constructors
HumanoidEnvV5(HumanoidEnvV5Options)
Builds the instance from its options record: the properties set, the constructor's defaults for the rest.
public HumanoidEnvV5(HumanoidEnvV5Options options)
Parameters
optionsHumanoidEnvV5OptionsThe options.
Exceptions
- ArgumentNullException
Thrown when
optionsis null.
HumanoidEnvV5(RenderMode, string, int, IReadOnlyDictionary<string, object?>?, double, double, double, (double Low, double High)?, double, bool, (double Low, double High)?, double, bool, bool, bool, bool, bool, int, int, int?, string?, int, IReadOnlyDictionary<int, bool>?, EnvViewerFactory?)
Builds the env with the render mode given as the RenderMode enum (DR-84, DR-93), as
HumanoidEnvV5(string, int, IReadOnlyDictionary<string, object?>?, double, double, double, (double Low, double High)?, double, bool, (double Low, double High)?, double, bool, bool, bool, bool, bool, string?, int, int, int?, string?, int, IReadOnlyDictionary<int, bool>?, EnvViewerFactory?)
builds it from the mode's string: new HumanoidEnvV5(RenderMode.RgbArray) builds what
new HumanoidEnvV5(renderMode: "rgb_array") builds, the member reaching the env as Gymnasium's string
(Name(RenderMode)), which its RenderMode property then holds.
public HumanoidEnvV5(RenderMode renderMode, string xmlFile = "humanoid.xml", int frameSkip = 5, IReadOnlyDictionary<string, object?>? defaultCameraConfig = null, double forwardRewardWeight = 1.25, double ctrlCostWeight = 0.1, double contactCostWeight = 5E-07, (double Low, double High)? contactCostRange = null, double healthyReward = 5, bool terminateWhenUnhealthy = true, (double Low, double High)? healthyZRange = null, double resetNoiseScale = 0.01, bool excludeCurrentPositionsFromObservation = true, bool includeCinertInObservation = true, bool includeCvelInObservation = true, bool includeQfrcActuatorInObservation = true, bool includeCfrcExtInObservation = true, int width = 480, int height = 480, int? cameraId = null, string? cameraName = null, int maxGeom = 1000, IReadOnlyDictionary<int, bool>? visualOptions = null, EnvViewerFactory? viewerFactory = null)
Parameters
renderModeRenderModeThe render mode, which the env must list in its
metadata["render_modes"]as for the string; it behaves exactly as its Gymnasium string does in the other constructor.xmlFilestringThe model: a bundled file name (
xml_file, DR-63).frameSkipintPhysics steps per env step (
frame_skip).defaultCameraConfigIReadOnlyDictionary<string, object>The free camera's settings (
default_camera_config); null is Gymnasium's default (tracking body 1 from 4 m, looking at (0, 0, 2), 20° from above), and an empty dictionary leaves MuJoCo's camera as it is.forwardRewardWeightdoubleThe weight of the forward velocity (
forward_reward_weight).ctrlCostWeightdoubleThe weight of the squared controls (
ctrl_cost_weight).contactCostWeightdoubleThe weight of the squared external forces (
contact_cost_weight).contactCostRange(double hx, double hy)?The range the contact cost is clipped to (
contact_cost_range); null is (-inf, 10.0).healthyRewarddoubleThe reward for each healthy step (
healthy_reward).terminateWhenUnhealthyboolWhether an unhealthy state ends the episode (
terminate_when_unhealthy).healthyZRange(double hx, double hy)?The exclusive range of a healthy torso height (
healthy_z_range); null is (1.0, 2.0).resetNoiseScaledoubleThe scale of the reset's noise (
reset_noise_scale).excludeCurrentPositionsFromObservationboolWhether the root's x and y leave the observation (
exclude_current_positions_from_observation).includeCinertInObservationboolWhether the bodies' inertias join the observation (
include_cinert_in_observation).includeCvelInObservationboolWhether the bodies' velocities join the observation (
include_cvel_in_observation).includeQfrcActuatorInObservationboolWhether the actuator forces join the observation (
include_qfrc_actuator_in_observation).includeCfrcExtInObservationboolWhether the external forces join the observation (
include_cfrc_ext_in_observation).widthintThe rendered frame's width (
width).heightintThe rendered frame's height (
height).cameraIdint?The camera to render from (
camera_id).cameraNamestringThe camera to render from by name (
camera_name).maxGeomintThe most geoms a frame draws (
max_geom).visualOptionsIReadOnlyDictionary<int, bool>MuJoCo visual flags to switch (
visual_options).viewerFactoryEnvViewerFactoryGym.NET's
viewer_factory(DR-20) forhumanmode.
Remarks
The mode is required and comes first, so new HumanoidEnvV5() keeps meaning the other constructor, without a
mode; leave the mode out there for no rendering. Every other parameter means what it means there.
Exceptions
- ArgumentOutOfRangeException
Thrown for a value RenderMode does not declare (a cast integer such as
(RenderMode)99).- NotSupportedException
xmlFileis not a bundled model.- OverflowError
A frame skip of 0 (Python's
OverflowErrorderiving the frame rate).- ValueError
Both a camera id and a camera name.
HumanoidEnvV5(string, int, IReadOnlyDictionary<string, object?>?, double, double, double, (double Low, double High)?, double, bool, (double Low, double High)?, double, bool, bool, bool, bool, bool, string?, int, int, int?, string?, int, IReadOnlyDictionary<int, bool>?, EnvViewerFactory?)
Creates the env (Gymnasium's HumanoidEnv.init): the model, its spaces and the frame rate the time step gives.
public HumanoidEnvV5(string xmlFile = "humanoid.xml", int frameSkip = 5, IReadOnlyDictionary<string, object?>? defaultCameraConfig = null, double forwardRewardWeight = 1.25, double ctrlCostWeight = 0.1, double contactCostWeight = 5E-07, (double Low, double High)? contactCostRange = null, double healthyReward = 5, bool terminateWhenUnhealthy = true, (double Low, double High)? healthyZRange = null, double resetNoiseScale = 0.01, bool excludeCurrentPositionsFromObservation = true, bool includeCinertInObservation = true, bool includeCvelInObservation = true, bool includeQfrcActuatorInObservation = true, bool includeCfrcExtInObservation = true, string? renderMode = null, int width = 480, int height = 480, int? cameraId = null, string? cameraName = null, int maxGeom = 1000, IReadOnlyDictionary<int, bool>? visualOptions = null, EnvViewerFactory? viewerFactory = null)
Parameters
xmlFilestringThe model: a bundled file name (
xml_file, DR-63).frameSkipintPhysics steps per env step (
frame_skip).defaultCameraConfigIReadOnlyDictionary<string, object>The free camera's settings (
default_camera_config); null is Gymnasium's default (tracking body 1 from 4 m, looking at (0, 0, 2), 20° from above), and an empty dictionary leaves MuJoCo's camera as it is.forwardRewardWeightdoubleThe weight of the forward velocity (
forward_reward_weight).ctrlCostWeightdoubleThe weight of the squared controls (
ctrl_cost_weight).contactCostWeightdoubleThe weight of the squared external forces (
contact_cost_weight).contactCostRange(double hx, double hy)?The range the contact cost is clipped to (
contact_cost_range); null is (-inf, 10.0).healthyRewarddoubleThe reward for each healthy step (
healthy_reward).terminateWhenUnhealthyboolWhether an unhealthy state ends the episode (
terminate_when_unhealthy).healthyZRange(double hx, double hy)?The exclusive range of a healthy torso height (
healthy_z_range); null is (1.0, 2.0).resetNoiseScaledoubleThe scale of the reset's noise (
reset_noise_scale).excludeCurrentPositionsFromObservationboolWhether the root's x and y leave the observation (
exclude_current_positions_from_observation).includeCinertInObservationboolWhether the bodies' inertias join the observation (
include_cinert_in_observation).includeCvelInObservationboolWhether the bodies' velocities join the observation (
include_cvel_in_observation).includeQfrcActuatorInObservationboolWhether the actuator forces join the observation (
include_qfrc_actuator_in_observation).includeCfrcExtInObservationboolWhether the external forces join the observation (
include_cfrc_ext_in_observation).renderModestringGymnasium's
render_mode:human,rgb_array,depth_array,rgbd_tupleor null.widthintThe rendered frame's width (
width).heightintThe rendered frame's height (
height).cameraIdint?The camera to render from (
camera_id).cameraNamestringThe camera to render from by name (
camera_name).maxGeomintThe most geoms a frame draws (
max_geom).visualOptionsIReadOnlyDictionary<int, bool>MuJoCo visual flags to switch (
visual_options).viewerFactoryEnvViewerFactoryGym.NET's
viewer_factory(DR-20) forhumanmode.
Exceptions
- NotSupportedException
xmlFileis not a bundled model.- OverflowError
A frame skip of 0 (Python's
OverflowErrorderiving the frame rate).- ValueError
Both a camera id and a camera name.
Properties
ClassMetadata
Gymnasium's class attribute metadata: the four render modes, the frame rate set per instance. A new dict on every read.
public static Metadata ClassMetadata { get; }
Property Value
ContactCost
The contact cost now (contact_cost): contact_cost_weight * sum(cfrc_ext**2) clipped to
ContactCostRange.
[PyName("contact_cost")]
public double ContactCost { get; }
Property Value
ContactCostRange
The range the contact cost is clipped to (_contact_cost_range).
[PyName("_contact_cost_range")]
public (double Low, double High) ContactCostRange { get; set; }
Property Value
ContactCostWeight
The weight of the squared external forces (_contact_cost_weight), read every step.
[PyName("_contact_cost_weight")]
public double ContactCostWeight { get; set; }
Property Value
CtrlCostWeight
The weight of the squared controls (_ctrl_cost_weight), read every step.
[PyName("_ctrl_cost_weight")]
public double CtrlCostWeight { get; set; }
Property Value
ExcludeCurrentPositionsFromObservation
Whether the root's x and y leave the observation (_exclude_current_positions_from_observation). Footgun:
this and the four Include... flags change the observation's length when changed after construction, but
not the observation space, as in Gymnasium.
[PyName("_exclude_current_positions_from_observation")]
public bool ExcludeCurrentPositionsFromObservation { get; set; }
Property Value
ForwardRewardWeight
The weight of the forward velocity (_forward_reward_weight), read every step.
[PyName("_forward_reward_weight")]
public double ForwardRewardWeight { get; set; }
Property Value
HealthyReward
The healthy reward now (healthy_reward): is_healthy * _healthy_reward, an np.float64 there.
[PyName("healthy_reward")]
public double HealthyReward { get; }
Property Value
HealthyRewardWeight
The reward for each healthy step (_healthy_reward), read every step.
[PyName("_healthy_reward")]
public double HealthyRewardWeight { get; set; }
Property Value
HealthyZRange
The exclusive range of a healthy torso height (_healthy_z_range).
[PyName("_healthy_z_range")]
public (double Low, double High) HealthyZRange { get; set; }
Property Value
IncludeCfrcExtInObservation
Whether the external forces join the observation (_include_cfrc_ext_in_observation).
[PyName("_include_cfrc_ext_in_observation")]
public bool IncludeCfrcExtInObservation { get; set; }
Property Value
IncludeCinertInObservation
Whether the bodies' inertias join the observation (_include_cinert_in_observation).
[PyName("_include_cinert_in_observation")]
public bool IncludeCinertInObservation { get; set; }
Property Value
IncludeCvelInObservation
Whether the bodies' velocities join the observation (_include_cvel_in_observation).
[PyName("_include_cvel_in_observation")]
public bool IncludeCvelInObservation { get; set; }
Property Value
IncludeQfrcActuatorInObservation
Whether the actuator forces join the observation (_include_qfrc_actuator_in_observation).
[PyName("_include_qfrc_actuator_in_observation")]
public bool IncludeQfrcActuatorInObservation { get; set; }
Property Value
IsHealthy
Whether the humanoid is healthy now (is_healthy): the torso's height inside the exclusive z range.
[PyName("is_healthy")]
public bool IsHealthy { get; }
Property Value
ObservationStructure
How the observation is laid out (observation_structure): skipped_qpos, qpos, qvel,
cinert, cvel, qfrc_actuator, cfrc_ext, and 0 for ten_length and
ten_velocity (reported in the info, not observed).
public Info ObservationStructure { get; }
Property Value
ResetNoiseScale
The scale of the reset's noise (_reset_noise_scale), read every reset.
[PyName("_reset_noise_scale")]
public double ResetNoiseScale { get; set; }
Property Value
TerminateWhenUnhealthy
Whether an unhealthy state ends the episode (_terminate_when_unhealthy), read every step.
[PyName("_terminate_when_unhealthy")]
public bool TerminateWhenUnhealthy { get; set; }
Property Value
Methods
ControlCost(NDArray?)
The control cost (control_cost): ctrl_cost_weight * sum(ctrl**2) of the simulation's controls,
float64; action is taken but not read, as in Gymnasium.
[PyName("control_cost")]
public double ControlCost(NDArray? action)
Parameters
actionNDArrayThe action; ignored.
Returns
- double
The cost.
GetResetInfo()
The reset's info (_get_reset_info): x_position, y_position, copies of
tendon_length and tendon_velocity, and distance_from_origin.
protected override Info GetResetInfo()
Returns
- Info
The info.
MassCenter(MjModel, MjData)
The center of mass's x and y (mass_center): einsum("b,bj->j", body_mass, xipos) / body_mass.sum(),
from the bodies' inertial frames of the last forward pass. A new (2,) float64 array the caller owns.
[PyName("mass_center")]
public static NDArray MassCenter(MjModel model, MjData data)
Parameters
Returns
- NDArray
The
(2,)array.
Remarks
Reference: gymnasium/envs/mujoco/humanoid_v5.py.
Exceptions
- ArgumentNullException
A null model or data.
ResetModel()
The episode's start (reset_model): positions init_qpos + uniform(-scale, scale, nq), then
velocities init_qvel + uniform(-scale, scale, nv), drawn in that order.
protected override NDArray ResetModel()
Returns
- NDArray
The first observation.
Step(NDArray)
Applies action for FrameSkip physics steps and reports the transition (step).
public override StepResult<NDArray> Step(NDArray action)
Parameters
actionNDArrayThe 17 controls, any real dtype (stored as float64).
Returns
- StepResult<NDArray>
The observation, the float64 reward, whether an unhealthy state ended the episode,
truncated=Falseand the info.
Exceptions
- ArgumentNullException
actionis null.- ValueError
The action is not of shape
(17,), with Gymnasium's message.