Table of Contents

Class AntEnvV5

Namespace
Gym.Environments.Envs.MuJoCo
Assembly
Gym.Environments.dll

Ant-v5: a four-legged ant (a torso and four two-link legs on a free joint) driven by 8 hip and ankle torques in [-1, 1]. The reward pays the main body's forward velocity and staying healthy, and charges the squared torques and the clipped contact forces; a non-finite state or a torso height outside [0.2, 1.0] ends the episode. Byte for byte what Gymnasium's Ant-v5 computes on MuJoCo 4.0.0 (DR-61, DR-64).

[PyName("AntEnv")]
public class AntEnvV5 : MujocoEnv, IEnv, IDisposable, IDynamicAttributes
Inheritance
AntEnvV5
Implements
Inherited Members
Extension Methods

Remarks

Observation (float64): the positions without the root's x and y (unless asked for), the velocities, then the contact forces of every body but the world clipped to contact_force_range (unless include_cfrc_ext_in_observation=False), 105 values by default. Reward: x_velocity * forward_reward_weight + healthy_reward * is_healthy - ctrl_cost_weight * sum(action**2) - contact_cost_weight * sum(clip(cfrc_ext)**2), the velocity measured on main_body's frame. Info: x_position, y_position, distance_from_origin (of the root joint), x_velocity, y_velocity, reward_forward, reward_ctrl, reward_contact, reward_survive; the reset's info holds the first three.

Not thread-safe. Reference: gymnasium/envs/mujoco/ant_v5.py.

Its info reads typed (DR-91): AntInfoV5 info = step.Info; (AntInfoV5), with the dict's own values still behind the indexer.

Constructors

AntEnvV5(AntEnvV5Options)

Builds the instance from its options record: the properties set, the constructor's defaults for the rest.

public AntEnvV5(AntEnvV5Options options)

Parameters

options AntEnvV5Options

The options.

Exceptions

ArgumentNullException

Thrown when options is null.

AntEnvV5(RenderMode, string, int, IReadOnlyDictionary<string, object?>?, double, double, double, double, object?, bool, (double Low, double High)?, (double Low, double High)?, double, bool, bool, int, int, int?, string?, int, IReadOnlyDictionary<int, bool>?, EnvViewerFactory?)

Builds the env with the render mode given as the RenderMode enum (DR-84, DR-93), as AntEnvV5(string, int, IReadOnlyDictionary<string, object?>?, double, double, double, double, object?, bool, (double Low, double High)?, (double Low, double High)?, double, bool, bool, string?, int, int, int?, string?, int, IReadOnlyDictionary<int, bool>?, EnvViewerFactory?) builds it from the mode's string: new AntEnvV5(RenderMode.RgbArray) builds what new AntEnvV5(renderMode: "rgb_array") builds, the member reaching the env as Gymnasium's string (Name(RenderMode)), which its RenderMode property then holds.

public AntEnvV5(RenderMode renderMode, string xmlFile = "ant.xml", int frameSkip = 5, IReadOnlyDictionary<string, object?>? defaultCameraConfig = null, double forwardRewardWeight = 1, double ctrlCostWeight = 0.5, double contactCostWeight = 0.0005, double healthyReward = 1, object? mainBody = null, bool terminateWhenUnhealthy = true, (double Low, double High)? healthyZRange = null, (double Low, double High)? contactForceRange = null, double resetNoiseScale = 0.1, bool excludeCurrentPositionsFromObservation = true, bool includeCfrcExtInObservation = true, int width = 480, int height = 480, int? cameraId = null, string? cameraName = null, int maxGeom = 1000, IReadOnlyDictionary<int, bool>? visualOptions = null, EnvViewerFactory? viewerFactory = null)

Parameters

renderMode RenderMode

The render mode, which the env must list in its metadata["render_modes"] as for the string; it behaves exactly as its Gymnasium string does in the other constructor.

xmlFile string

The model: a bundled file name (xml_file, DR-63).

frameSkip int

Physics steps per env step (frame_skip).

defaultCameraConfig IReadOnlyDictionary<string, object>

The free camera's settings (default_camera_config); null is Gymnasium's default, {"distance": 4.0}, and an empty dictionary leaves MuJoCo's camera as it is.

forwardRewardWeight double

The weight of the forward velocity (forward_reward_weight; Gymnasium's default is the int 1).

ctrlCostWeight double

The weight of the squared torques (ctrl_cost_weight).

contactCostWeight double

The weight of the squared clipped contact forces (contact_cost_weight).

healthyReward double

The reward for each healthy step (healthy_reward).

mainBody object

The body whose motion is the forward velocity (main_body): its index (an integer, a bool or a 0-d integer array) or its name; null is Gymnasium's default, body 1 (the torso). Resolved at each step, as Gymnasium resolves it, so a bad one fails at the first step with MuJoCo's error.

terminateWhenUnhealthy bool

Whether an unhealthy state ends the episode (terminate_when_unhealthy).

healthyZRange (double hx, double hy)?

The inclusive range of a healthy torso height (healthy_z_range); null is (0.2, 1.0).

contactForceRange (double hx, double hy)?

The range the contact forces are clipped to (contact_force_range); null is (-1.0, 1.0).

resetNoiseScale double

The scale of the reset's noise (reset_noise_scale).

excludeCurrentPositionsFromObservation bool

Whether the root's x and y leave the observation (exclude_current_positions_from_observation).

includeCfrcExtInObservation bool

Whether the clipped contact forces join the observation (include_cfrc_ext_in_observation).

width int

The rendered frame's width (width).

height int

The rendered frame's height (height).

cameraId int?

The camera to render from (camera_id).

cameraName string

The camera to render from by name (camera_name).

maxGeom int

The most geoms a frame draws (max_geom).

visualOptions IReadOnlyDictionary<int, bool>

MuJoCo visual flags to switch (visual_options).

viewerFactory EnvViewerFactory

Gym.NET's viewer_factory (DR-20) for human mode.

Remarks

The mode is required and comes first, so new AntEnvV5() keeps meaning the other constructor, without a mode; leave the mode out there for no rendering. Every other parameter means what it means there.

Exceptions

ArgumentOutOfRangeException

Thrown for a value RenderMode does not declare (a cast integer such as (RenderMode)99).

NotSupportedException

xmlFile is not a bundled model.

OverflowError

A frame skip of 0 (Python's OverflowError deriving the frame rate).

ValueError

Both a camera id and a camera name.

AntEnvV5(string, int, IReadOnlyDictionary<string, object?>?, double, double, double, double, object?, bool, (double Low, double High)?, (double Low, double High)?, double, bool, bool, string?, int, int, int?, string?, int, IReadOnlyDictionary<int, bool>?, EnvViewerFactory?)

Creates the env (Gymnasium's AntEnv.init): the model, its spaces and the frame rate the time step gives.

public AntEnvV5(string xmlFile = "ant.xml", int frameSkip = 5, IReadOnlyDictionary<string, object?>? defaultCameraConfig = null, double forwardRewardWeight = 1, double ctrlCostWeight = 0.5, double contactCostWeight = 0.0005, double healthyReward = 1, object? mainBody = null, bool terminateWhenUnhealthy = true, (double Low, double High)? healthyZRange = null, (double Low, double High)? contactForceRange = null, double resetNoiseScale = 0.1, bool excludeCurrentPositionsFromObservation = true, bool includeCfrcExtInObservation = true, string? renderMode = null, int width = 480, int height = 480, int? cameraId = null, string? cameraName = null, int maxGeom = 1000, IReadOnlyDictionary<int, bool>? visualOptions = null, EnvViewerFactory? viewerFactory = null)

Parameters

xmlFile string

The model: a bundled file name (xml_file, DR-63).

frameSkip int

Physics steps per env step (frame_skip).

defaultCameraConfig IReadOnlyDictionary<string, object>

The free camera's settings (default_camera_config); null is Gymnasium's default, {"distance": 4.0}, and an empty dictionary leaves MuJoCo's camera as it is.

forwardRewardWeight double

The weight of the forward velocity (forward_reward_weight; Gymnasium's default is the int 1).

ctrlCostWeight double

The weight of the squared torques (ctrl_cost_weight).

contactCostWeight double

The weight of the squared clipped contact forces (contact_cost_weight).

healthyReward double

The reward for each healthy step (healthy_reward).

mainBody object

The body whose motion is the forward velocity (main_body): its index (an integer, a bool or a 0-d integer array) or its name; null is Gymnasium's default, body 1 (the torso). Resolved at each step, as Gymnasium resolves it, so a bad one fails at the first step with MuJoCo's error.

terminateWhenUnhealthy bool

Whether an unhealthy state ends the episode (terminate_when_unhealthy).

healthyZRange (double hx, double hy)?

The inclusive range of a healthy torso height (healthy_z_range); null is (0.2, 1.0).

contactForceRange (double hx, double hy)?

The range the contact forces are clipped to (contact_force_range); null is (-1.0, 1.0).

resetNoiseScale double

The scale of the reset's noise (reset_noise_scale).

excludeCurrentPositionsFromObservation bool

Whether the root's x and y leave the observation (exclude_current_positions_from_observation).

includeCfrcExtInObservation bool

Whether the clipped contact forces join the observation (include_cfrc_ext_in_observation).

renderMode string

Gymnasium's render_mode: human, rgb_array, depth_array, rgbd_tuple or null.

width int

The rendered frame's width (width).

height int

The rendered frame's height (height).

cameraId int?

The camera to render from (camera_id).

cameraName string

The camera to render from by name (camera_name).

maxGeom int

The most geoms a frame draws (max_geom).

visualOptions IReadOnlyDictionary<int, bool>

MuJoCo visual flags to switch (visual_options).

viewerFactory EnvViewerFactory

Gym.NET's viewer_factory (DR-20) for human mode.

Exceptions

NotSupportedException

xmlFile is not a bundled model.

OverflowError

A frame skip of 0 (Python's OverflowError deriving the frame rate).

ValueError

Both a camera id and a camera name.

Properties

ClassMetadata

Gymnasium's class attribute metadata: the four render modes, the frame rate set per instance. A new dict on every read.

public static Metadata ClassMetadata { get; }

Property Value

Metadata

ContactCost

The contact cost now (contact_cost): contact_cost_weight * sum(contact_forces**2).

[PyName("contact_cost")]
public double ContactCost { get; }

Property Value

double

ContactCostWeight

The weight of the squared clipped contact forces (_contact_cost_weight), read every step.

[PyName("_contact_cost_weight")]
public double ContactCostWeight { get; set; }

Property Value

double

ContactForceRange

The range the contact forces are clipped to (_contact_force_range), for the observation and the cost.

[PyName("_contact_force_range")]
public (double Low, double High) ContactForceRange { get; set; }

Property Value

(double From, double To)

ContactForces

The contact forces clipped to ContactForceRange (contact_forces), one row of 6 per body (the world's included). A new (nbody, 6) array on every read, which the caller owns.

[PyName("contact_forces")]
public NDArray ContactForces { get; }

Property Value

NDArray

CtrlCostWeight

The weight of the squared torques (_ctrl_cost_weight), read every step.

[PyName("_ctrl_cost_weight")]
public double CtrlCostWeight { get; set; }

Property Value

double

ExcludeCurrentPositionsFromObservation

Whether the root's x and y leave the observation (_exclude_current_positions_from_observation). Footgun: changing it after construction changes the observation's length but not the observation space, as in Gymnasium.

[PyName("_exclude_current_positions_from_observation")]
public bool ExcludeCurrentPositionsFromObservation { get; set; }

Property Value

bool

ForwardRewardWeight

The weight of the forward velocity (_forward_reward_weight), read every step.

[PyName("_forward_reward_weight")]
public double ForwardRewardWeight { get; set; }

Property Value

double

HealthyReward

The healthy reward now (healthy_reward): is_healthy * _healthy_reward, an np.float64 there.

[PyName("healthy_reward")]
public double HealthyReward { get; }

Property Value

double

HealthyRewardWeight

The reward for each healthy step (_healthy_reward), read every step.

[PyName("_healthy_reward")]
public double HealthyRewardWeight { get; set; }

Property Value

double

HealthyZRange

The inclusive range of a healthy torso height (_healthy_z_range).

[PyName("_healthy_z_range")]
public (double Low, double High) HealthyZRange { get; set; }

Property Value

(double From, double To)

IncludeCfrcExtInObservation

Whether the clipped contact forces join the observation (_include_cfrc_ext_in_observation).

[PyName("_include_cfrc_ext_in_observation")]
public bool IncludeCfrcExtInObservation { get; set; }

Property Value

bool

IsHealthy

Whether the ant is healthy now (is_healthy): every position and velocity finite and the torso's height inside the inclusive z range.

[PyName("is_healthy")]
public bool IsHealthy { get; }

Property Value

bool

MainBody

The body whose motion is the forward velocity (_main_body): an index or a name, resolved at each step.

[PyName("_main_body")]
public object? MainBody { get; set; }

Property Value

object

ObservationStructure

How the observation is laid out (observation_structure): skipped_qpos, qpos, qvel and cfrc_ext counts.

public Info ObservationStructure { get; }

Property Value

Info

ResetNoiseScale

The scale of the reset's noise (_reset_noise_scale), read every reset.

[PyName("_reset_noise_scale")]
public double ResetNoiseScale { get; set; }

Property Value

double

TerminateWhenUnhealthy

Whether an unhealthy state ends the episode (_terminate_when_unhealthy), read every step.

[PyName("_terminate_when_unhealthy")]
public bool TerminateWhenUnhealthy { get; set; }

Property Value

bool

Methods

ControlCost(NDArray)

The control cost of action (control_cost): ctrl_cost_weight * sum(action**2), computed in the action's dtype and returned widened exactly.

[PyName("control_cost")]
public double ControlCost(NDArray action)

Parameters

action NDArray

The action.

Returns

double

The cost.

Exceptions

ArgumentNullException

action is null.

NotSupportedException

An action of a dtype whose cost NumPy keeps in that dtype (float16, complex).

GetResetInfo()

The reset's info (_get_reset_info): x_position, y_position and distance_from_origin of the root joint, np.float64s.

protected override Info GetResetInfo()

Returns

Info

The info.

ResetModel()

The episode's start (reset_model): positions init_qpos + uniform(-scale, scale, nq), then velocities init_qvel + scale * standard_normal(nv), drawn in that order.

protected override NDArray ResetModel()

Returns

NDArray

The first observation.

Step(NDArray)

Applies action for FrameSkip physics steps and reports the transition (step).

public override StepResult<NDArray> Step(NDArray action)

Parameters

action NDArray

The 8 torques, float32 (the action space's samples) or any real dtype.

Returns

StepResult<NDArray>

The observation, the float64 reward, whether an unhealthy state ended the episode, truncated=False and the info.

Exceptions

ArgumentNullException

action is null.

ValueError

The action is not of shape (8,), with Gymnasium's message.

NotSupportedException

An action of a dtype whose cost NumPy keeps in that dtype (float16, complex).

KeyError

MainBody names no body (MuJoCo's message).

IndexError

MainBody is an index outside the model's bodies (MuJoCo's message).

TypeError

MainBody is neither an integer nor a string (pybind11's message).