Class AntEnvV4
- Namespace
- Gym.Environments.Envs.MuJoCo
- Assembly
- Gym.Environments.dll
Ant-v4: the four-legged ant of AntEnvV5 with v4's frame skip, the torso's velocity as the forward
reward (unweighted), v4's healthy reward (paid on every step while terminate_when_unhealthy is set), no
contact forces unless use_contact_forces=True, and an empty reset info. Byte for byte what Gymnasium's
Ant-v4 computes on MuJoCo 4.0.0 (DR-61, DR-64).
[PyName("AntEnv")]
public class AntEnvV4 : MujocoEnv, IEnv, IDisposable, IDynamicAttributes
- Inheritance
-
AntEnvV4
- Implements
- Inherited Members
- Extension Methods
Remarks
Observation (float64): the positions without the root's x and y (unless asked for), the velocities, and with
use_contact_forces the clipped contact forces of every body, the world's included; the space declares
27 values (+2 with the root's x and y, +84 with contact forces) whatever the model, as v4 does. Reward:
x_velocity + float(is_healthy or terminate_when_unhealthy) * healthy_reward - ctrl_cost_weight *
sum(action**2), minus the contact cost with use_contact_forces, which then also replaces the info's
reward_ctrl with -contact_cost (Gymnasium's own quirk, kept). make warns that the id is out
of date.
Not thread-safe. Reference: gymnasium/envs/mujoco/ant_v4.py.
Its info reads typed (DR-91): AntInfoV4 info = step.Info; (AntInfoV4), with the dict's own values still behind the indexer.
Constructors
AntEnvV4(AntEnvV4Options)
Builds the instance from its options record: the properties set, the constructor's defaults for the rest.
public AntEnvV4(AntEnvV4Options options)
Parameters
optionsAntEnvV4OptionsThe options.
Exceptions
- ArgumentNullException
Thrown when
optionsis null.
AntEnvV4(RenderMode, string, double, bool, double, double, bool, (double Low, double High)?, (double Low, double High)?, double, bool, int, int, int?, string?, int, IReadOnlyDictionary<int, bool>?, EnvViewerFactory?)
Builds the env with the render mode given as the RenderMode enum (DR-84, DR-93), as
AntEnvV4(string, double, bool, double, double, bool, (double Low, double High)?, (double Low, double High)?, double, bool, string?, int, int, int?, string?, int, IReadOnlyDictionary<int, bool>?, EnvViewerFactory?)
builds it from the mode's string: new AntEnvV4(RenderMode.RgbArray) builds what
new AntEnvV4(renderMode: "rgb_array") builds, the member reaching the env as Gymnasium's string
(Name(RenderMode)), which its RenderMode property then holds.
public AntEnvV4(RenderMode renderMode, string xmlFile = "ant.xml", double ctrlCostWeight = 0.5, bool useContactForces = false, double contactCostWeight = 0.0005, double healthyReward = 1, bool terminateWhenUnhealthy = true, (double Low, double High)? healthyZRange = null, (double Low, double High)? contactForceRange = null, double resetNoiseScale = 0.1, bool excludeCurrentPositionsFromObservation = true, int width = 480, int height = 480, int? cameraId = null, string? cameraName = null, int maxGeom = 1000, IReadOnlyDictionary<int, bool>? visualOptions = null, EnvViewerFactory? viewerFactory = null)
Parameters
renderModeRenderModeThe render mode, which the env must list in its
metadata["render_modes"]as for the string; it behaves exactly as its Gymnasium string does in the other constructor.xmlFilestringThe model: a bundled file name (
xml_file, DR-63).ctrlCostWeightdoubleThe weight of the squared torques (
ctrl_cost_weight).useContactForcesboolWhether the clipped contact forces join the observation and the cost (
use_contact_forces).contactCostWeightdoubleThe weight of the squared clipped contact forces (
contact_cost_weight).healthyRewarddoubleThe healthy reward per step (
healthy_reward).terminateWhenUnhealthyboolWhether an unhealthy state ends the episode (
terminate_when_unhealthy).healthyZRange(double hx, double hy)?The inclusive range of a healthy torso height (
healthy_z_range); null is (0.2, 1.0).contactForceRange(double hx, double hy)?The range the contact forces are clipped to (
contact_force_range); null is (-1.0, 1.0).resetNoiseScaledoubleThe scale of the reset's noise (
reset_noise_scale).excludeCurrentPositionsFromObservationboolWhether the root's x and y leave the observation (
exclude_current_positions_from_observation).widthintThe rendered frame's width (
width).heightintThe rendered frame's height (
height).cameraIdint?The camera to render from (
camera_id).cameraNamestringThe camera to render from by name (
camera_name).maxGeomintThe most geoms a frame draws (
max_geom).visualOptionsIReadOnlyDictionary<int, bool>MuJoCo visual flags to switch (
visual_options).viewerFactoryEnvViewerFactoryGym.NET's
viewer_factory(DR-20) forhumanmode.
Remarks
The mode is required and comes first, so new AntEnvV4() keeps meaning the other constructor, without a
mode; leave the mode out there for no rendering. Every other parameter means what it means there.
Exceptions
- ArgumentOutOfRangeException
Thrown for a value RenderMode does not declare (a cast integer such as
(RenderMode)99).- NotSupportedException
xmlFileis not a bundled model.- AssertionError
A model whose time step does not give v4's 20 frames per second (Gymnasium's assertion).
- ValueError
Both a camera id and a camera name.
AntEnvV4(string, double, bool, double, double, bool, (double Low, double High)?, (double Low, double High)?, double, bool, string?, int, int, int?, string?, int, IReadOnlyDictionary<int, bool>?, EnvViewerFactory?)
Creates the env (Gymnasium's AntEnv.init): the model with a frame skip of 5.
public AntEnvV4(string xmlFile = "ant.xml", double ctrlCostWeight = 0.5, bool useContactForces = false, double contactCostWeight = 0.0005, double healthyReward = 1, bool terminateWhenUnhealthy = true, (double Low, double High)? healthyZRange = null, (double Low, double High)? contactForceRange = null, double resetNoiseScale = 0.1, bool excludeCurrentPositionsFromObservation = true, string? renderMode = null, int width = 480, int height = 480, int? cameraId = null, string? cameraName = null, int maxGeom = 1000, IReadOnlyDictionary<int, bool>? visualOptions = null, EnvViewerFactory? viewerFactory = null)
Parameters
xmlFilestringThe model: a bundled file name (
xml_file, DR-63).ctrlCostWeightdoubleThe weight of the squared torques (
ctrl_cost_weight).useContactForcesboolWhether the clipped contact forces join the observation and the cost (
use_contact_forces).contactCostWeightdoubleThe weight of the squared clipped contact forces (
contact_cost_weight).healthyRewarddoubleThe healthy reward per step (
healthy_reward).terminateWhenUnhealthyboolWhether an unhealthy state ends the episode (
terminate_when_unhealthy).healthyZRange(double hx, double hy)?The inclusive range of a healthy torso height (
healthy_z_range); null is (0.2, 1.0).contactForceRange(double hx, double hy)?The range the contact forces are clipped to (
contact_force_range); null is (-1.0, 1.0).resetNoiseScaledoubleThe scale of the reset's noise (
reset_noise_scale).excludeCurrentPositionsFromObservationboolWhether the root's x and y leave the observation (
exclude_current_positions_from_observation).renderModestringGymnasium's
render_mode:human,rgb_array,depth_array,rgbd_tupleor null.widthintThe rendered frame's width (
width).heightintThe rendered frame's height (
height).cameraIdint?The camera to render from (
camera_id).cameraNamestringThe camera to render from by name (
camera_name).maxGeomintThe most geoms a frame draws (
max_geom).visualOptionsIReadOnlyDictionary<int, bool>MuJoCo visual flags to switch (
visual_options).viewerFactoryEnvViewerFactoryGym.NET's
viewer_factory(DR-20) forhumanmode.
Exceptions
- NotSupportedException
xmlFileis not a bundled model.- AssertionError
A model whose time step does not give v4's 20 frames per second (Gymnasium's assertion).
- ValueError
Both a camera id and a camera name.
Properties
ClassMetadata
Gymnasium's class attribute metadata: the four render modes at 20 frames per second. A new dict on every read.
public static Metadata ClassMetadata { get; }
Property Value
ContactCost
The contact cost now (contact_cost): contact_cost_weight * sum(contact_forces**2).
[PyName("contact_cost")]
public double ContactCost { get; }
Property Value
ContactCostWeight
The weight of the squared clipped contact forces (_contact_cost_weight), read every step.
[PyName("_contact_cost_weight")]
public double ContactCostWeight { get; set; }
Property Value
ContactForceRange
The range the contact forces are clipped to (_contact_force_range).
[PyName("_contact_force_range")]
public (double Low, double High) ContactForceRange { get; set; }
Property Value
ContactForces
The contact forces clipped to ContactForceRange (contact_forces), one row of 6 per body
(the world's included). A new (nbody, 6) array on every read, which the caller owns.
[PyName("contact_forces")]
public NDArray ContactForces { get; }
Property Value
CtrlCostWeight
The weight of the squared torques (_ctrl_cost_weight), read every step.
[PyName("_ctrl_cost_weight")]
public double CtrlCostWeight { get; set; }
Property Value
ExcludeCurrentPositionsFromObservation
Whether the root's x and y leave the observation (_exclude_current_positions_from_observation). Footgun:
changing it after construction changes the observation's length but not the observation space, as in Gymnasium.
[PyName("_exclude_current_positions_from_observation")]
public bool ExcludeCurrentPositionsFromObservation { get; set; }
Property Value
HealthyReward
The healthy reward now (healthy_reward): float(is_healthy or terminate_when_unhealthy) *
_healthy_reward, so paid on every step while unhealthy states end the episode.
[PyName("healthy_reward")]
public double HealthyReward { get; }
Property Value
HealthyRewardWeight
The healthy reward per step (_healthy_reward), read every step.
[PyName("_healthy_reward")]
public double HealthyRewardWeight { get; set; }
Property Value
HealthyZRange
The inclusive range of a healthy torso height (_healthy_z_range).
[PyName("_healthy_z_range")]
public (double Low, double High) HealthyZRange { get; set; }
Property Value
IsHealthy
Whether the ant is healthy now (is_healthy): every position and velocity finite and the torso's height
inside the inclusive z range.
[PyName("is_healthy")]
public bool IsHealthy { get; }
Property Value
ResetNoiseScale
The scale of the reset's noise (_reset_noise_scale), read every reset.
[PyName("_reset_noise_scale")]
public double ResetNoiseScale { get; set; }
Property Value
TerminateWhenUnhealthy
Whether an unhealthy state ends the episode (_terminate_when_unhealthy), read every step.
[PyName("_terminate_when_unhealthy")]
public bool TerminateWhenUnhealthy { get; set; }
Property Value
Terminated
Whether the episode ended (terminated): unhealthy while terminate_when_unhealthy is set.
[PyName("terminated")]
public bool Terminated { get; }
Property Value
UseContactForces
Whether the clipped contact forces join the observation and the cost (_use_contact_forces). Footgun:
changing it after construction changes the observation's length but not the observation space, as in Gymnasium.
[PyName("_use_contact_forces")]
public bool UseContactForces { get; set; }
Property Value
Methods
ControlCost(NDArray)
The control cost of action (control_cost): ctrl_cost_weight * sum(action**2),
computed in the action's dtype and returned widened exactly.
[PyName("control_cost")]
public double ControlCost(NDArray action)
Parameters
actionNDArrayThe action.
Returns
- double
The cost.
Exceptions
- ArgumentNullException
actionis null.- NotSupportedException
An action of a dtype whose cost NumPy keeps in that dtype (float16, complex).
ResetModel()
The episode's start (reset_model): positions init_qpos + uniform(-scale, scale, nq), then
velocities init_qvel + scale * standard_normal(nv), drawn in that order.
protected override NDArray ResetModel()
Returns
- NDArray
The first observation.
Step(NDArray)
Applies action for 5 physics steps and reports the transition (step).
public override StepResult<NDArray> Step(NDArray action)
Parameters
actionNDArrayThe 8 torques, float32 (the action space's samples) or any real dtype.
Returns
- StepResult<NDArray>
The observation, the float64 reward, Terminated,
truncated=Falseand the info.
Exceptions
- ArgumentNullException
actionis null.- ValueError
The action is not of shape
(8,), with Gymnasium's message.- NotSupportedException
An action of a dtype whose cost NumPy keeps in that dtype (float16, complex).
- KeyError
The model has no body named
torso(MuJoCo's message).