Class AsyncVectorEnv<TObs, TAct>
A vector env that runs its sub-envs in parallel, each call on a worker thread (Gymnasium's AsyncVectorEnv,
whose workers are processes): the same results as SyncVectorEnv<TObs, TAct> gathered in sub-env order,
for envs whose steps are long enough to be worth a thread. Typed by the batched types; AsyncVectorEnv
is the untyped one.
public class AsyncVectorEnv<TObs, TAct> : VectorEnv<TObs, TAct>, IVectorEnv, IDisposable
Type Parameters
TObsThe batched observation type: the element type of the single observation space batched.
TActThe batched action type.
- Inheritance
-
VectorEnv<TObs, TAct>AsyncVectorEnv<TObs, TAct>
- Implements
- Derived
- Inherited Members
- Extension Methods
Remarks
Gymnasium's protocol stays: ResetAsync(BigInteger?, IReadOnlyDictionary<string, object?>?),
StepAsync(TAct) and CallAsync(string, params object?[]) send a call to every sub-env and return at once; the matching
ResetWait(double?), StepWait(double?) and CallWait(double?) collect the results, optionally within a
timeout (TimeoutException, Python's multiprocessing.TimeoutError); one call is pending at a
time (AsyncState), and Reset(BigInteger?, IReadOnlyDictionary<string, object?>?) and
Step(TAct) are the two halves back to back. Each sub-env is built by its worker from EnvFns,
owned by it and closed by it, after one extra build of the first function in the constructor to read the metadata
and spaces, as Gymnasium's dummy env.
Where the calls run is ExecutionMode (DR-22): the thread pool, dedicated threads shared by the
sub-envs, or one thread per sub-env; a sub-env rendering to a window gets its own thread in pool mode. Results are
gathered in sub-env order whatever finishes first, so every mode gives the same bits. MaxConcurrency
limits how many sub-envs run reset or step at once. With SharedMemory (the default)
the workers write their observations straight into one buffer (CreateSharedMemory(Space, int)), as
Gymnasium's processes write into shared memory; threads share memory anyway, so the option changes the copy path,
not the results, except where Gymnasium's own two paths differ (see the deviations).
A sub-env that raises ends its worker, as Gymnasium's process does: the wait that collects the call logs Gymnasium's
three error lines (the worker's index, the exception with its .NET stack trace where Python prints its traceback,
and the last line) and rethrows the last failing sub-env's exception as it was thrown. That sub-env is closed and
detached, so any later call that needs it fails as Python's does on the dropped pipe (AttributeError
'NoneType' object has no attribute 'send'); closing still works. Errors are collected in the order the
sub-envs failed, as Python's error queue receives them.
Ownership (D19): with Copy every observation batch returned is a new deep copy the caller owns; without it, the batch is this env's own buffer (the shared one when SharedMemory is on), overwritten by the next call. Rewards and flags are always new arrays. Not thread-safe: one thread drives the vector env.
Footgun: the sub-envs run at once on different threads of one process, where Gymnasium's processes each receive a copy of whatever the functions capture. A function must therefore build everything mutable its sub-env uses (a space, a generator, a buffer) rather than share one object among sub-envs: two threads sampling one space race on its generator. The actions given to StepAsync(TAct) are read by the workers after it returns; keep them until the matching wait.
Deviations, each where Python's processes have no thread counterpart: context is accepted and has nothing
to choose (threads have no start method); daemon decides whether the dedicated threads are background
threads (pool threads always are); a custom worker (AsyncVectorWorker, DR-38) runs on a thread
of its own over a PipeConnection pair, whatever the execution mode; a sub-env function that throws
makes the constructor throw that exception (Python's parent sees its worker's pipe close), except under a custom
worker, where the parent's receive reports the end of the stream as Python's does
(EndOfStreamException, carrying the exception); close(terminate=True) cannot kill a thread, so it stops waiting and lets
each worker close its sub-env as soon as the call it runs returns, and Dispose() waits for
that; a full reset clears a NEXT_STEP worker's pending autoreset in both copy paths (DR-42), where
Gymnasium's worker clears it only with shared memory, so its next step without shared memory resets that sub-env
a second time, with a reward of 0 (Gym.NET's sync and async envs agree in every sequence instead). Kept from
Gymnasium on purpose: a Text or OneOf shared buffer is read once, at construction; a
SAME_STEP info puts final_info before
final_obs; reset_mask leaves the flags of the unmasked sub-envs alone. As in
SyncVectorEnv<TObs, TAct>, a reset_mask stays in the caller's options dict, and
NpRandom and NpRandomSeed keep the base's
meaning (GetAttr("np_random") gives the sub-envs').
Reference: gymnasium/vector/async_vector_env.py.
Constructors
AsyncVectorEnv(IEnumerable<Func<IEnv>>, bool, bool, object?, object?, object?, AsyncExecutionMode, int?, string?, bool, AsyncVectorWorker?)
Builds the vector env (AsyncVectorEnv(env_fns, shared_memory, copy, observation_mode=..., autoreset_mode=...,
max_concurrency=...)): the first function once more for the metadata and spaces, then one worker per function,
each building its sub-env on its own thread, and the check that every sub-env's spaces fit the batch.
public AsyncVectorEnv(IEnumerable<Func<IEnv>> envFns, bool sharedMemory = true, bool copy = true, object? observationMode = null, object? autoresetMode = null, object? maxConcurrency = null, AsyncExecutionMode executionMode = AsyncExecutionMode.ThreadPool, int? threadCount = null, string? context = null, bool daemon = true, AsyncVectorWorker? worker = null)
Parameters
envFnsIEnumerable<Func<IEnv>>One function per sub-env; each is called by its worker, which owns and closes what it returns. The first is also called once by the constructor (closed at once).
sharedMemoryboolWhether the workers write their observations into one shared buffer (the default); a space with no shared layout is refused when on.
copyboolWhether every returned observation batch is a deep copy (the default) or the env's own buffer.
observationModeobject"same"(null, the default): every sub-env has the first's observation space, batched;"different": the spaces may differ but share shape and dtype (each function is called once more to read its space); or a pair (object?[] or a value tuple) of the batched and the single space to use.autoresetModeobjectAn AutoresetMode or its value as a string; null is NextStep.
maxConcurrencyobjectHow many sub-envs may run
resetorstepat once: a positive integer (any .NET integer or a 0-d integer array), or null for no limit.executionModeAsyncExecutionModeWhere the calls run (DR-22); ThreadPool by default.
threadCountint?The number of threads of DedicatedThreads (default ProcessorCount); refused in the other modes.
contextstringGymnasium's
multiprocessingstart method ("spawn","fork", ...): accepted and kept, with nothing to choose, since the workers are threads.daemonboolWhether the dedicated threads (of DedicatedThreads, ThreadPerEnv, a thread-bound sub-env, a custom worker) are background threads, which never keep the process alive (the default, as Gymnasium's daemon processes); false makes them foreground threads the process waits for. Pool threads are background threads either way.
workerAsyncVectorWorkerA custom worker (DR-38) to serve each sub-env instead of the built-in one, on a thread of its own over a pipe, whatever
executionModesays; usually a wrapper around AsyncWorker(int, Func<IEnv>, PipeConnection, PipeConnection, object?, ConcurrentQueue<AsyncWorkerError>, AutoresetMode, SemaphoreSlim?). null (the default) keeps the built-in workers ofexecutionMode.
Exceptions
- ArgumentNullException
Thrown when
envFnsis null.- ValueError
Thrown for an unknown autoreset or observation mode, a
maxConcurrencythat is not a positive integer, a misplaced or non-positivethreadCount, or a shared buffer for a space that has none, with Gymnasium's messages.- IndexError
Thrown when there are no sub-envs (
list index out of range).- TypeError
Thrown when a pair observation mode holds something else than spaces, a batched space's elements are not
TObs/TAct, or a shared buffer is asked for aGraphorSequencespace.- RuntimeError
Thrown when a sub-env's spaces do not fit the batch, with Gymnasium's messages.
- EndOfStreamException
Thrown under a custom worker that ended without answering (a sub-env function that throws, carried as the inner exception), as Python's parent sees
EOFError.- OSError
Thrown under a custom worker that ended before the first command reached it (
Broken pipe, Python'sBrokenPipeError), carrying its exception likewise.- Exception
Whatever a sub-env function throws, rethrown after the other workers were shut down.
Properties
AutoresetMode
When ended sub-envs reset (autoreset_mode), also in metadata["autoreset_mode"]; applied by each worker.
public AutoresetMode AutoresetMode { get; }
Property Value
Context
The start method given (context), kept with nothing to choose: the workers are threads.
public string? Context { get; }
Property Value
Copy
Whether returned observation batches are deep copies (copy).
public bool Copy { get; }
Property Value
Daemon
Whether the dedicated threads are background threads (daemon); pool threads always are.
public bool Daemon { get; }
Property Value
EnvFns
The functions the sub-envs are built from (env_fns), in order.
public IReadOnlyList<Func<IEnv>> EnvFns { get; }
Property Value
ExecutionMode
Where the workers' calls run (DR-22).
public AsyncExecutionMode ExecutionMode { get; }
Property Value
MaxConcurrency
How many sub-envs may run reset or step at once (max_concurrency), or null for no limit; other calls are not limited.
public int? MaxConcurrency { get; }
Property Value
- int?
ObservationMode
The observation mode given (observation_mode): "same", "different", or the pair of batched and single spaces.
public object ObservationMode { get; }
Property Value
SharedMemory
Whether the workers write their observations into one shared buffer (shared_memory).
public bool SharedMemory { get; }
Property Value
State
The pending call (Python's _state): Default when none.
public AsyncState State { get; }
Property Value
ThreadCount
The number of dedicated threads in DedicatedThreads mode, or null in the other modes.
public int? ThreadCount { get; }
Property Value
- int?
WorkerFunction
The custom worker serving each sub-env (worker, DR-38), or null for the built-in workers.
[PyName("worker")]
public AsyncVectorWorker? WorkerFunction { get; }
Property Value
Methods
Call(string, params object?[])
Calls a sub-env member by its Python name on every sub-env, in parallel, and waits (call(name, *args)):
an attribute's value (a delegate one is called with the arguments), or a method, called with the arguments.
public IReadOnlyList<object?> Call(string name, params object?[] args)
Parameters
namestringThe Python name:
render,gravity, a run-time attribute.argsobject[]The positional arguments for a callable member.
Returns
- IReadOnlyList<object>
One result per sub-env, in order.
Exceptions
- ClosedEnvironmentError
Thrown after Close().
- AlreadyPendingCallError
Thrown while another call is pending.
- Exception
The last failing sub-env's exception (an unknown attribute's AttributeError,
reset/step/closerefused with ValueError), after the error lines are logged.
CallAsync(string, params object?[])
Sends a member call to every sub-env and returns at once (call_async(name, *args)); CallWait(double?) collects the results.
public void CallAsync(string name, params object?[] args)
Parameters
Exceptions
- ClosedEnvironmentError
Thrown after Close().
- AlreadyPendingCallError
Thrown while another call is pending:
Callingcall_asyncwhile waiting for a pending call tostepto complete.- AttributeError
Thrown when a sub-env's worker ended with an error earlier.
CallWait(double?)
Collects the results of CallAsync(string, params object?[]) (call_wait(timeout)).
public IReadOnlyList<object?> CallWait(double? timeout = null)
Parameters
Returns
- IReadOnlyList<object>
One result per sub-env, in order.
Exceptions
- ClosedEnvironmentError
Thrown after Close().
- NoAsyncCallError
Thrown when no call is pending:
Callingcall_waitwithout any prior call tocall_async.- TimeoutException
Thrown when a sub-env did not answer in time; the state returns to Default.
- AttributeError
Thrown when a sub-env's worker ended with an error earlier.
- Exception
The last failing sub-env's exception, after the error lines are logged.
Close(double?, bool)
Closes the vector env (close(timeout=..., terminate=...)): a pending call is collected first (within
timeout, with Gymnasium's warning), then every worker closes its sub-env and ends.
public void Close(double? timeout, bool terminate = false)
Parameters
timeoutdouble?Seconds to wait for a pending call; when it runs out, the close becomes a terminating one.
terminateboolWhether to stop at once: Python kills its workers; threads cannot be killed, so each worker closes its sub-env as soon as the call it is running returns, and this returns without waiting for that (disposal waits).
Exceptions
- Exception
A pending call's sub-env exception, as Gymnasium's close lets it through (the env stays open).
CloseExtras()
Closes without a timeout or termination: what Close() runs (close_extras()).
protected override void CloseExtras()
Dispose(bool)
Closes (Python's del closes an open one with terminate=True), waits for every worker to close its
sub-env, then releases the threads, the buffers and the spaces this env made.
protected override void Dispose(bool disposing)
Parameters
GetAttr(string)
Reads an attribute of every sub-env (get_attr(name)), as Call(string, params object?[]) does without arguments.
public IReadOnlyList<object?> GetAttr(string name)
Parameters
namestringThe Python name.
Returns
- IReadOnlyList<object>
One value per sub-env.
Exceptions
- ClosedEnvironmentError
Thrown after Close().
- AlreadyPendingCallError
Thrown while another call is pending.
- Exception
The last failing sub-env's exception (an unknown attribute's AttributeError), after the error lines are logged.
Render()
Renders every sub-env (render(), Gymnasium's call("render")), in parallel.
public override IReadOnlyList<RenderResult>? Render()
Returns
- IReadOnlyList<RenderResult>
One result per sub-env, each by its own render mode.
Exceptions
- ClosedEnvironmentError
Thrown after Close().
- AlreadyPendingCallError
Thrown while another call is pending.
Reset(IReadOnlyList<BigInteger?>, IReadOnlyDictionary<string, object?>?)
Resets every sub-env with a seed each and waits for them (reset(seed=[...])).
public override VectorResetResult<TObs> Reset(IReadOnlyList<BigInteger?> seeds, IReadOnlyDictionary<string, object?>? options = null)
Parameters
seedsIReadOnlyList<BigInteger?>One seed per sub-env, null keeping that sub-env's generator.
optionsIReadOnlyDictionary<string, object>The options passed to every reset sub-env, with an optional
reset_mask.
Returns
- VectorResetResult<TObs>
The batched observations and the merged infos.
Exceptions
- ArgumentNullException
Thrown when
seedsis null.- ValueError
Thrown when there is not one seed per sub-env, with Gymnasium's message, or for a bad
reset_mask.- ClosedEnvironmentError
Thrown after Close().
- AlreadyPendingCallError
Thrown while another call is pending.
- TypeError
Thrown for a
reset_maskthat is not a bool array.- Exception
The last failing sub-env's exception, after the error lines are logged.
Reset(BigInteger?, IReadOnlyDictionary<string, object?>?)
Resets every sub-env in parallel and waits for them (reset(seed=seed, options=options)).
public override VectorResetResult<TObs> Reset(BigInteger? seed = null, IReadOnlyDictionary<string, object?>? options = null)
Parameters
seedBigInteger?The first sub-env's seed (sub-env
igetsseed + i), or null to keep every generator.optionsIReadOnlyDictionary<string, object>The options passed to every reset sub-env; a
reset_maskentry resets only the masked ones.
Returns
- VectorResetResult<TObs>
The batched observations (a copy unless Copy is off) and the merged infos.
Exceptions
- ClosedEnvironmentError
Thrown after Close().
- AlreadyPendingCallError
Thrown while another call is pending.
- TypeError
Thrown when
reset_maskis not an array or not bool.- ValueError
Thrown when
reset_maskhas another shape or no true.- Exception
The last failing sub-env's exception, after the error lines are logged.
ResetAsync(IReadOnlyList<BigInteger?>, IReadOnlyDictionary<string, object?>?)
Sends reset with a seed per sub-env and returns at once (reset_async(seed=[...])).
public void ResetAsync(IReadOnlyList<BigInteger?> seeds, IReadOnlyDictionary<string, object?>? options = null)
Parameters
seedsIReadOnlyList<BigInteger?>One seed per sub-env.
optionsIReadOnlyDictionary<string, object>The options, with an optional
reset_mask.
Exceptions
- ArgumentNullException
Thrown when
seedsis null.- ClosedEnvironmentError
Thrown after Close().
- ValueError
Thrown when there is not one seed per sub-env (Gymnasium's message, before the pending-call check), or for a bad
reset_mask.- AlreadyPendingCallError
Thrown while another call is pending.
- TypeError
Thrown for a
reset_maskthat is not a bool array.- AttributeError
Thrown when a sub-env's worker ended with an error earlier.
ResetAsync(BigInteger?, IReadOnlyDictionary<string, object?>?)
Sends reset to every sub-env and returns at once (reset_async(seed, options)); ResetWait(double?)
collects the results. With options["reset_mask"], the masked-out sub-envs answer with their last
observation instead.
public void ResetAsync(BigInteger? seed = null, IReadOnlyDictionary<string, object?>? options = null)
Parameters
seedBigInteger?The first sub-env's seed (sub-env
igetsseed + i), or null.optionsIReadOnlyDictionary<string, object>The options, with an optional
reset_mask(a(num_envs,)bool array with a true).
Exceptions
- ClosedEnvironmentError
Thrown after Close().
- AlreadyPendingCallError
Thrown while another call is pending:
Calling.reset_asyncwhile waiting for a pending call tostepto complete- TypeError
Thrown when
reset_maskis not an array or not bool.- ValueError
Thrown when
reset_maskhas another shape or no true.- AttributeError
Thrown when a sub-env's worker ended with an error earlier (the pipe Python dropped).
ResetWait(double?)
Collects the results of ResetAsync(BigInteger?, IReadOnlyDictionary<string, object?>?) (reset_wait(timeout)).
public VectorResetResult<TObs> ResetWait(double? timeout = null)
Parameters
Returns
- VectorResetResult<TObs>
The batched observations (a copy unless Copy is off) and the merged infos.
Exceptions
- ClosedEnvironmentError
Thrown after Close().
- NoAsyncCallError
Thrown when no reset is pending:
Callingreset_waitwithout any prior call toreset_async.- TimeoutException
Thrown when a sub-env did not answer in time; the call stays with the workers, and the state returns to Default.
- AttributeError
Thrown when a sub-env's worker ended with an error earlier.
- Exception
The last failing sub-env's exception, after the error lines are logged.
SetAttr(string, object?)
Sets an attribute of every sub-env through its wrappers and waits (set_attr(name, values)): a list or tuple
gives one value per sub-env, anything else is set on all of them.
public void SetAttr(string name, object? values)
Parameters
namestringThe Python name.
valuesobjectA list or object?[] of one value per sub-env, or one value for all (an array is one value).
Exceptions
- ClosedEnvironmentError
Thrown after Close().
- ValueError
Thrown when a list's length is not the number of sub-envs, with Gymnasium's message.
- AlreadyPendingCallError
Thrown while another call is pending.
- Exception
The last failing sub-env's exception, after the error lines are logged.
Step(TAct)
Steps every sub-env in parallel and waits for them (step(actions)).
public override VectorStepResult<TObs> Step(TAct actions)
Parameters
actionsTActThe batched actions, one per sub-env, split by the batched action space (
iterate).
Returns
- VectorStepResult<TObs>
The batched observations (a copy unless Copy is off), new reward and flag arrays, and the merged infos.
Exceptions
- ClosedEnvironmentError
Thrown after Close().
- AlreadyPendingCallError
Thrown while another call is pending.
- ValueError
Thrown when the actions are not one per sub-env (Python's
zip(strict=True)text), after the paired sub-envs were sent theirs.- Exception
The last failing sub-env's exception, after the error lines are logged.
StepAsync(TAct)
Sends each sub-env its action and returns at once (step_async(actions)); StepWait(double?) collects the results.
public void StepAsync(TAct actions)
Parameters
actionsTActThe batched actions, split by the batched action space; read by the workers after this returns, so keep them until the wait.
Exceptions
- ClosedEnvironmentError
Thrown after Close().
- AlreadyPendingCallError
Thrown while another call is pending:
Callingstep_asyncwhile waiting for a pending call toresetto complete.- ValueError
Thrown when the actions are not one per sub-env, after the paired sub-envs were sent theirs (their results then wait in the workers' queues, as in Python's pipes).
- AttributeError
Thrown when a sub-env's worker ended with an error earlier.
StepWait(double?)
Collects the results of StepAsync(TAct) (step_wait(timeout)).
public VectorStepResult<TObs> StepWait(double? timeout = null)
Parameters
Returns
- VectorStepResult<TObs>
The batched observations (a copy unless Copy is off), new float64 rewards and bool flags, and the merged infos.
Exceptions
- ClosedEnvironmentError
Thrown after Close().
- NoAsyncCallError
Thrown when no step is pending:
Callingstep_waitwithout any prior call tostep_async.- TimeoutException
Thrown when a sub-env did not answer in time; the state returns to Default.
- AttributeError
Thrown when a sub-env's worker ended with an error earlier.
- Exception
The last failing sub-env's exception, after the error lines are logged.