Table of Contents

Class AsyncVectorEnv<TObs, TAct>

Namespace
Gym.Vector
Assembly
Gym.dll

A vector env that runs its sub-envs in parallel, each call on a worker thread (Gymnasium's AsyncVectorEnv, whose workers are processes): the same results as SyncVectorEnv<TObs, TAct> gathered in sub-env order, for envs whose steps are long enough to be worth a thread. Typed by the batched types; AsyncVectorEnv is the untyped one.

public class AsyncVectorEnv<TObs, TAct> : VectorEnv<TObs, TAct>, IVectorEnv, IDisposable

Type Parameters

TObs

The batched observation type: the element type of the single observation space batched.

TAct

The batched action type.

Inheritance
VectorEnv<TObs, TAct>
AsyncVectorEnv<TObs, TAct>
Implements
Derived
Inherited Members
Extension Methods

Remarks

Gymnasium's protocol stays: ResetAsync(BigInteger?, IReadOnlyDictionary<string, object?>?), StepAsync(TAct) and CallAsync(string, params object?[]) send a call to every sub-env and return at once; the matching ResetWait(double?), StepWait(double?) and CallWait(double?) collect the results, optionally within a timeout (TimeoutException, Python's multiprocessing.TimeoutError); one call is pending at a time (AsyncState), and Reset(BigInteger?, IReadOnlyDictionary<string, object?>?) and Step(TAct) are the two halves back to back. Each sub-env is built by its worker from EnvFns, owned by it and closed by it, after one extra build of the first function in the constructor to read the metadata and spaces, as Gymnasium's dummy env.

Where the calls run is ExecutionMode (DR-22): the thread pool, dedicated threads shared by the sub-envs, or one thread per sub-env; a sub-env rendering to a window gets its own thread in pool mode. Results are gathered in sub-env order whatever finishes first, so every mode gives the same bits. MaxConcurrency limits how many sub-envs run reset or step at once. With SharedMemory (the default) the workers write their observations straight into one buffer (CreateSharedMemory(Space, int)), as Gymnasium's processes write into shared memory; threads share memory anyway, so the option changes the copy path, not the results, except where Gymnasium's own two paths differ (see the deviations).

A sub-env that raises ends its worker, as Gymnasium's process does: the wait that collects the call logs Gymnasium's three error lines (the worker's index, the exception with its .NET stack trace where Python prints its traceback, and the last line) and rethrows the last failing sub-env's exception as it was thrown. That sub-env is closed and detached, so any later call that needs it fails as Python's does on the dropped pipe (AttributeError 'NoneType' object has no attribute 'send'); closing still works. Errors are collected in the order the sub-envs failed, as Python's error queue receives them.

Ownership (D19): with Copy every observation batch returned is a new deep copy the caller owns; without it, the batch is this env's own buffer (the shared one when SharedMemory is on), overwritten by the next call. Rewards and flags are always new arrays. Not thread-safe: one thread drives the vector env.

Footgun: the sub-envs run at once on different threads of one process, where Gymnasium's processes each receive a copy of whatever the functions capture. A function must therefore build everything mutable its sub-env uses (a space, a generator, a buffer) rather than share one object among sub-envs: two threads sampling one space race on its generator. The actions given to StepAsync(TAct) are read by the workers after it returns; keep them until the matching wait.

Deviations, each where Python's processes have no thread counterpart: context is accepted and has nothing to choose (threads have no start method); daemon decides whether the dedicated threads are background threads (pool threads always are); a custom worker (AsyncVectorWorker, DR-38) runs on a thread of its own over a PipeConnection pair, whatever the execution mode; a sub-env function that throws makes the constructor throw that exception (Python's parent sees its worker's pipe close), except under a custom worker, where the parent's receive reports the end of the stream as Python's does (EndOfStreamException, carrying the exception); close(terminate=True) cannot kill a thread, so it stops waiting and lets each worker close its sub-env as soon as the call it runs returns, and Dispose() waits for that; a full reset clears a NEXT_STEP worker's pending autoreset in both copy paths (DR-42), where Gymnasium's worker clears it only with shared memory, so its next step without shared memory resets that sub-env a second time, with a reward of 0 (Gym.NET's sync and async envs agree in every sequence instead). Kept from Gymnasium on purpose: a Text or OneOf shared buffer is read once, at construction; a SAME_STEP info puts final_info before final_obs; reset_mask leaves the flags of the unmasked sub-envs alone. As in SyncVectorEnv<TObs, TAct>, a reset_mask stays in the caller's options dict, and NpRandom and NpRandomSeed keep the base's meaning (GetAttr("np_random") gives the sub-envs').

Reference: gymnasium/vector/async_vector_env.py.

Constructors

AsyncVectorEnv(IEnumerable<Func<IEnv>>, bool, bool, object?, object?, object?, AsyncExecutionMode, int?, string?, bool, AsyncVectorWorker?)

Builds the vector env (AsyncVectorEnv(env_fns, shared_memory, copy, observation_mode=..., autoreset_mode=..., max_concurrency=...)): the first function once more for the metadata and spaces, then one worker per function, each building its sub-env on its own thread, and the check that every sub-env's spaces fit the batch.

public AsyncVectorEnv(IEnumerable<Func<IEnv>> envFns, bool sharedMemory = true, bool copy = true, object? observationMode = null, object? autoresetMode = null, object? maxConcurrency = null, AsyncExecutionMode executionMode = AsyncExecutionMode.ThreadPool, int? threadCount = null, string? context = null, bool daemon = true, AsyncVectorWorker? worker = null)

Parameters

envFns IEnumerable<Func<IEnv>>

One function per sub-env; each is called by its worker, which owns and closes what it returns. The first is also called once by the constructor (closed at once).

sharedMemory bool

Whether the workers write their observations into one shared buffer (the default); a space with no shared layout is refused when on.

copy bool

Whether every returned observation batch is a deep copy (the default) or the env's own buffer.

observationMode object

"same" (null, the default): every sub-env has the first's observation space, batched; "different": the spaces may differ but share shape and dtype (each function is called once more to read its space); or a pair (object?[] or a value tuple) of the batched and the single space to use.

autoresetMode object

An AutoresetMode or its value as a string; null is NextStep.

maxConcurrency object

How many sub-envs may run reset or step at once: a positive integer (any .NET integer or a 0-d integer array), or null for no limit.

executionMode AsyncExecutionMode

Where the calls run (DR-22); ThreadPool by default.

threadCount int?

The number of threads of DedicatedThreads (default ProcessorCount); refused in the other modes.

context string

Gymnasium's multiprocessing start method ("spawn", "fork", ...): accepted and kept, with nothing to choose, since the workers are threads.

daemon bool

Whether the dedicated threads (of DedicatedThreads, ThreadPerEnv, a thread-bound sub-env, a custom worker) are background threads, which never keep the process alive (the default, as Gymnasium's daemon processes); false makes them foreground threads the process waits for. Pool threads are background threads either way.

worker AsyncVectorWorker

A custom worker (DR-38) to serve each sub-env instead of the built-in one, on a thread of its own over a pipe, whatever executionMode says; usually a wrapper around AsyncWorker(int, Func<IEnv>, PipeConnection, PipeConnection, object?, ConcurrentQueue<AsyncWorkerError>, AutoresetMode, SemaphoreSlim?). null (the default) keeps the built-in workers of executionMode.

Exceptions

ArgumentNullException

Thrown when envFns is null.

ValueError

Thrown for an unknown autoreset or observation mode, a maxConcurrency that is not a positive integer, a misplaced or non-positive threadCount, or a shared buffer for a space that has none, with Gymnasium's messages.

IndexError

Thrown when there are no sub-envs (list index out of range).

TypeError

Thrown when a pair observation mode holds something else than spaces, a batched space's elements are not TObs/TAct, or a shared buffer is asked for a Graph or Sequence space.

RuntimeError

Thrown when a sub-env's spaces do not fit the batch, with Gymnasium's messages.

EndOfStreamException

Thrown under a custom worker that ended without answering (a sub-env function that throws, carried as the inner exception), as Python's parent sees EOFError.

OSError

Thrown under a custom worker that ended before the first command reached it (Broken pipe, Python's BrokenPipeError), carrying its exception likewise.

Exception

Whatever a sub-env function throws, rethrown after the other workers were shut down.

Properties

AutoresetMode

When ended sub-envs reset (autoreset_mode), also in metadata["autoreset_mode"]; applied by each worker.

public AutoresetMode AutoresetMode { get; }

Property Value

AutoresetMode

Context

The start method given (context), kept with nothing to choose: the workers are threads.

public string? Context { get; }

Property Value

string

Copy

Whether returned observation batches are deep copies (copy).

public bool Copy { get; }

Property Value

bool

Daemon

Whether the dedicated threads are background threads (daemon); pool threads always are.

public bool Daemon { get; }

Property Value

bool

EnvFns

The functions the sub-envs are built from (env_fns), in order.

public IReadOnlyList<Func<IEnv>> EnvFns { get; }

Property Value

IReadOnlyList<Func<IEnv>>

ExecutionMode

Where the workers' calls run (DR-22).

public AsyncExecutionMode ExecutionMode { get; }

Property Value

AsyncExecutionMode

MaxConcurrency

How many sub-envs may run reset or step at once (max_concurrency), or null for no limit; other calls are not limited.

public int? MaxConcurrency { get; }

Property Value

int?

ObservationMode

The observation mode given (observation_mode): "same", "different", or the pair of batched and single spaces.

public object ObservationMode { get; }

Property Value

object

SharedMemory

Whether the workers write their observations into one shared buffer (shared_memory).

public bool SharedMemory { get; }

Property Value

bool

State

The pending call (Python's _state): Default when none.

public AsyncState State { get; }

Property Value

AsyncState

ThreadCount

The number of dedicated threads in DedicatedThreads mode, or null in the other modes.

public int? ThreadCount { get; }

Property Value

int?

WorkerFunction

The custom worker serving each sub-env (worker, DR-38), or null for the built-in workers.

[PyName("worker")]
public AsyncVectorWorker? WorkerFunction { get; }

Property Value

AsyncVectorWorker

Methods

Call(string, params object?[])

Calls a sub-env member by its Python name on every sub-env, in parallel, and waits (call(name, *args)): an attribute's value (a delegate one is called with the arguments), or a method, called with the arguments.

public IReadOnlyList<object?> Call(string name, params object?[] args)

Parameters

name string

The Python name: render, gravity, a run-time attribute.

args object[]

The positional arguments for a callable member.

Returns

IReadOnlyList<object>

One result per sub-env, in order.

Exceptions

ClosedEnvironmentError

Thrown after Close().

AlreadyPendingCallError

Thrown while another call is pending.

Exception

The last failing sub-env's exception (an unknown attribute's AttributeError, reset/step/close refused with ValueError), after the error lines are logged.

CallAsync(string, params object?[])

Sends a member call to every sub-env and returns at once (call_async(name, *args)); CallWait(double?) collects the results.

public void CallAsync(string name, params object?[] args)

Parameters

name string

The Python name.

args object[]

The positional arguments.

Exceptions

ClosedEnvironmentError

Thrown after Close().

AlreadyPendingCallError

Thrown while another call is pending: Calling call_async while waiting for a pending call to step to complete.

AttributeError

Thrown when a sub-env's worker ended with an error earlier.

CallWait(double?)

Collects the results of CallAsync(string, params object?[]) (call_wait(timeout)).

public IReadOnlyList<object?> CallWait(double? timeout = null)

Parameters

timeout double?

Seconds to wait for every sub-env, or null to wait as long as it takes.

Returns

IReadOnlyList<object>

One result per sub-env, in order.

Exceptions

ClosedEnvironmentError

Thrown after Close().

NoAsyncCallError

Thrown when no call is pending: Calling call_wait without any prior call to call_async.

TimeoutException

Thrown when a sub-env did not answer in time; the state returns to Default.

AttributeError

Thrown when a sub-env's worker ended with an error earlier.

Exception

The last failing sub-env's exception, after the error lines are logged.

Close(double?, bool)

Closes the vector env (close(timeout=..., terminate=...)): a pending call is collected first (within timeout, with Gymnasium's warning), then every worker closes its sub-env and ends.

public void Close(double? timeout, bool terminate = false)

Parameters

timeout double?

Seconds to wait for a pending call; when it runs out, the close becomes a terminating one.

terminate bool

Whether to stop at once: Python kills its workers; threads cannot be killed, so each worker closes its sub-env as soon as the call it is running returns, and this returns without waiting for that (disposal waits).

Exceptions

Exception

A pending call's sub-env exception, as Gymnasium's close lets it through (the env stays open).

CloseExtras()

Closes without a timeout or termination: what Close() runs (close_extras()).

protected override void CloseExtras()

Dispose(bool)

Closes (Python's del closes an open one with terminate=True), waits for every worker to close its sub-env, then releases the threads, the buffers and the spaces this env made.

protected override void Dispose(bool disposing)

Parameters

disposing bool

true from Dispose().

GetAttr(string)

Reads an attribute of every sub-env (get_attr(name)), as Call(string, params object?[]) does without arguments.

public IReadOnlyList<object?> GetAttr(string name)

Parameters

name string

The Python name.

Returns

IReadOnlyList<object>

One value per sub-env.

Exceptions

ClosedEnvironmentError

Thrown after Close().

AlreadyPendingCallError

Thrown while another call is pending.

Exception

The last failing sub-env's exception (an unknown attribute's AttributeError), after the error lines are logged.

Render()

Renders every sub-env (render(), Gymnasium's call("render")), in parallel.

public override IReadOnlyList<RenderResult>? Render()

Returns

IReadOnlyList<RenderResult>

One result per sub-env, each by its own render mode.

Exceptions

ClosedEnvironmentError

Thrown after Close().

AlreadyPendingCallError

Thrown while another call is pending.

Reset(IReadOnlyList<BigInteger?>, IReadOnlyDictionary<string, object?>?)

Resets every sub-env with a seed each and waits for them (reset(seed=[...])).

public override VectorResetResult<TObs> Reset(IReadOnlyList<BigInteger?> seeds, IReadOnlyDictionary<string, object?>? options = null)

Parameters

seeds IReadOnlyList<BigInteger?>

One seed per sub-env, null keeping that sub-env's generator.

options IReadOnlyDictionary<string, object>

The options passed to every reset sub-env, with an optional reset_mask.

Returns

VectorResetResult<TObs>

The batched observations and the merged infos.

Exceptions

ArgumentNullException

Thrown when seeds is null.

ValueError

Thrown when there is not one seed per sub-env, with Gymnasium's message, or for a bad reset_mask.

ClosedEnvironmentError

Thrown after Close().

AlreadyPendingCallError

Thrown while another call is pending.

TypeError

Thrown for a reset_mask that is not a bool array.

Exception

The last failing sub-env's exception, after the error lines are logged.

Reset(BigInteger?, IReadOnlyDictionary<string, object?>?)

Resets every sub-env in parallel and waits for them (reset(seed=seed, options=options)).

public override VectorResetResult<TObs> Reset(BigInteger? seed = null, IReadOnlyDictionary<string, object?>? options = null)

Parameters

seed BigInteger?

The first sub-env's seed (sub-env i gets seed + i), or null to keep every generator.

options IReadOnlyDictionary<string, object>

The options passed to every reset sub-env; a reset_mask entry resets only the masked ones.

Returns

VectorResetResult<TObs>

The batched observations (a copy unless Copy is off) and the merged infos.

Exceptions

ClosedEnvironmentError

Thrown after Close().

AlreadyPendingCallError

Thrown while another call is pending.

TypeError

Thrown when reset_mask is not an array or not bool.

ValueError

Thrown when reset_mask has another shape or no true.

Exception

The last failing sub-env's exception, after the error lines are logged.

ResetAsync(IReadOnlyList<BigInteger?>, IReadOnlyDictionary<string, object?>?)

Sends reset with a seed per sub-env and returns at once (reset_async(seed=[...])).

public void ResetAsync(IReadOnlyList<BigInteger?> seeds, IReadOnlyDictionary<string, object?>? options = null)

Parameters

seeds IReadOnlyList<BigInteger?>

One seed per sub-env.

options IReadOnlyDictionary<string, object>

The options, with an optional reset_mask.

Exceptions

ArgumentNullException

Thrown when seeds is null.

ClosedEnvironmentError

Thrown after Close().

ValueError

Thrown when there is not one seed per sub-env (Gymnasium's message, before the pending-call check), or for a bad reset_mask.

AlreadyPendingCallError

Thrown while another call is pending.

TypeError

Thrown for a reset_mask that is not a bool array.

AttributeError

Thrown when a sub-env's worker ended with an error earlier.

ResetAsync(BigInteger?, IReadOnlyDictionary<string, object?>?)

Sends reset to every sub-env and returns at once (reset_async(seed, options)); ResetWait(double?) collects the results. With options["reset_mask"], the masked-out sub-envs answer with their last observation instead.

public void ResetAsync(BigInteger? seed = null, IReadOnlyDictionary<string, object?>? options = null)

Parameters

seed BigInteger?

The first sub-env's seed (sub-env i gets seed + i), or null.

options IReadOnlyDictionary<string, object>

The options, with an optional reset_mask (a (num_envs,) bool array with a true).

Exceptions

ClosedEnvironmentError

Thrown after Close().

AlreadyPendingCallError

Thrown while another call is pending: Calling reset_async while waiting for a pending call to step to complete.

TypeError

Thrown when reset_mask is not an array or not bool.

ValueError

Thrown when reset_mask has another shape or no true.

AttributeError

Thrown when a sub-env's worker ended with an error earlier (the pipe Python dropped).

ResetWait(double?)

Collects the results of ResetAsync(BigInteger?, IReadOnlyDictionary<string, object?>?) (reset_wait(timeout)).

public VectorResetResult<TObs> ResetWait(double? timeout = null)

Parameters

timeout double?

Seconds to wait for every sub-env, or null to wait as long as it takes.

Returns

VectorResetResult<TObs>

The batched observations (a copy unless Copy is off) and the merged infos.

Exceptions

ClosedEnvironmentError

Thrown after Close().

NoAsyncCallError

Thrown when no reset is pending: Calling reset_wait without any prior call to reset_async.

TimeoutException

Thrown when a sub-env did not answer in time; the call stays with the workers, and the state returns to Default.

AttributeError

Thrown when a sub-env's worker ended with an error earlier.

Exception

The last failing sub-env's exception, after the error lines are logged.

SetAttr(string, object?)

Sets an attribute of every sub-env through its wrappers and waits (set_attr(name, values)): a list or tuple gives one value per sub-env, anything else is set on all of them.

public void SetAttr(string name, object? values)

Parameters

name string

The Python name.

values object

A list or object?[] of one value per sub-env, or one value for all (an array is one value).

Exceptions

ClosedEnvironmentError

Thrown after Close().

ValueError

Thrown when a list's length is not the number of sub-envs, with Gymnasium's message.

AlreadyPendingCallError

Thrown while another call is pending.

Exception

The last failing sub-env's exception, after the error lines are logged.

Step(TAct)

Steps every sub-env in parallel and waits for them (step(actions)).

public override VectorStepResult<TObs> Step(TAct actions)

Parameters

actions TAct

The batched actions, one per sub-env, split by the batched action space (iterate).

Returns

VectorStepResult<TObs>

The batched observations (a copy unless Copy is off), new reward and flag arrays, and the merged infos.

Exceptions

ClosedEnvironmentError

Thrown after Close().

AlreadyPendingCallError

Thrown while another call is pending.

ValueError

Thrown when the actions are not one per sub-env (Python's zip(strict=True) text), after the paired sub-envs were sent theirs.

Exception

The last failing sub-env's exception, after the error lines are logged.

StepAsync(TAct)

Sends each sub-env its action and returns at once (step_async(actions)); StepWait(double?) collects the results.

public void StepAsync(TAct actions)

Parameters

actions TAct

The batched actions, split by the batched action space; read by the workers after this returns, so keep them until the wait.

Exceptions

ClosedEnvironmentError

Thrown after Close().

AlreadyPendingCallError

Thrown while another call is pending: Calling step_async while waiting for a pending call to reset to complete.

ValueError

Thrown when the actions are not one per sub-env, after the paired sub-envs were sent theirs (their results then wait in the workers' queues, as in Python's pipes).

AttributeError

Thrown when a sub-env's worker ended with an error earlier.

StepWait(double?)

Collects the results of StepAsync(TAct) (step_wait(timeout)).

public VectorStepResult<TObs> StepWait(double? timeout = null)

Parameters

timeout double?

Seconds to wait for every sub-env, or null to wait as long as it takes.

Returns

VectorStepResult<TObs>

The batched observations (a copy unless Copy is off), new float64 rewards and bool flags, and the merged infos.

Exceptions

ClosedEnvironmentError

Thrown after Close().

NoAsyncCallError

Thrown when no step is pending: Calling step_wait without any prior call to step_async.

TimeoutException

Thrown when a sub-env did not answer in time; the state returns to Default.

AttributeError

Thrown when a sub-env's worker ended with an error earlier.

Exception

The last failing sub-env's exception, after the error lines are logged.