Table of Contents

Class CmaEsSampler

Namespace
OptunaSharp.Samplers
Assembly
OptunaSharp.dll

A sampler using cmaes as the backend. Port of optuna/samplers/_cmaes.py: CmaEsSampler driving the OptunaSharp.Samplers.CmaEs.Cma optimizer.

This sampler does not support CategoricalDistribution. If the search space includes categorical parameters they are sampled by the independent sampler.

Optimizer state is Optuna's pickle. Like Optuna, the sampler stores pickle.dumps(optimizer).hex() of a cmaes CMA, SepCMA or CMAwM under the chunked cma:optimizer:{i} (sepcma:, cmawm:) system attrs, written and read by OptunaSharp.Samplers.CmaEs.CmaPickle, so C# and Python workers can share one CMA-ES study in any order. A stored state is byte-identical to the one cmaes 0.13.1 stores for the same optimizer, except the _funhist_values entries cmaes leaves uninitialized (np.empty) until its first generations fill them. States written by OptunaSharp before 2026-10-29 (a cma\n-tagged JSON) are still read; Python cannot read those, so a study gains Python workers once a current C# worker has stored the next state. With with_margin=True the Python workers need cmaes 0.13.1, whose CMAwM state OptunaSharp reads.

Faster than Optuna's structure, same results. Optuna restores the optimizer from its pickle and rescans every trial on every ask, which makes a run quadratic in its trial count. This sampler keeps the restored optimizer for the rest of its generation and folds the history incrementally (CmaTrialHistory), reading an in-memory study through its zero-copy view; seeded trials and every stored byte stay identical to Optuna's (Rustuna plan 13, phase 1; the measured gain is in docs/plans/13-rustuna-derived-performance.md).

public class CmaEsSampler : BaseSampler
Inheritance
CmaEsSampler
Inherited Members

Examples

var sampler = new CmaEsSampler();
var study = Optuna.CreateStudy(sampler: sampler);
study.Optimize(t => Math.Pow(t.SuggestFloat("x", -1, 1), 2) + t.SuggestInt("y", -1, 1), nTrials: 20);

Constructors

CmaEsSampler(IReadOnlyDictionary<string, object?>?, double?, int, BaseSampler?, bool, int?, bool, string?, int?, int, bool, bool, bool, List<FrozenTrial>?, bool)

Configures covariance-matrix adaptation and its independent startup and unsupported-parameter fallback.

public CmaEsSampler(IReadOnlyDictionary<string, object?>? x0 = null, double? sigma0 = null, int nStartupTrials = 1, BaseSampler? independentSampler = null, bool warnIndependentSampling = true, int? seed = null, bool considerPrunedTrials = false, string? restartStrategy = null, int? popsize = null, int incPopsize = -1, bool useSeparableCma = false, bool withMargin = false, bool lrAdapt = false, List<FrozenTrial>? sourceTrials = null, bool reuseToldOptimizer = false)

Parameters

x0 IReadOnlyDictionary<string, object>

Initial parameter values for CMA-ES (external representation). By default the mean of the low and high of each distribution. Deprecated in v4.9.0.

sigma0 double?

Initial standard deviation of CMA-ES. By default min_range / 6. Deprecated in v4.9.0.

nStartupTrials int

Independent sampling is used until this many trials finish.

independentSampler BaseSampler

Sampler for parameters outside the relative search space (default RandomSampler).

warnIndependentSampling bool

Emit a warning when a parameter is sampled independently.

seed int?

Random seed for CMA-ES.

considerPrunedTrials bool

If true, PRUNED trials are considered for sampling (experimental).

restartStrategy string

Deprecated in v4.4.0 — falls back to null.

popsize int?

Population size of CMA-ES.

incPopsize int

Deprecated in v4.4.0 — no longer used.

useSeparableCma bool

Constrain the covariance matrix to be diagonal (experimental).

withMargin bool

Use CMA-ES with margin for discrete distributions (experimental).

lrAdapt bool

Use CMA-ES with learning-rate adaptation (experimental).

sourceTrials List<FrozenTrial>

Warm Starting CMA-ES source trials (experimental).

reuseToldOptimizer bool

Off by default; turning it on makes seeded runs stop matching Python Optuna (from about the third generation; same solution quality, a different run). When on, the optimizer that just told a generation keeps answering the asks that follow, as Rustuna keeps its live optimizer, instead of being restored from the state it stored: about 15 % faster on a 40-D study. The state is still stored every generation exactly as Optuna stores it, so resuming a study and Python workers on the same storage keep working; they just do not reproduce this worker's samples. An OptunaSharp extension with no Optuna counterpart, so it comes after every Optuna parameter.

Exceptions

ValueError

Source-trial initialization, explicit initial parameters, separable adaptation, margin correction, or learning-rate adaptation are combined incompatibly.

Methods

AfterTrial(Study, FrozenTrial, TrialState, IReadOnlyList<double>?)

Trial post-processing: called after the objective function returns and right before the trial is finished and its state is stored.

public override void AfterTrial(Study study, FrozenTrial trial, TrialState state, IReadOnlyList<double>? values)

Parameters

study Study

Study that owns the trial being finished.

trial FrozenTrial

Trial snapshot before its terminal state is written to storage.

state TrialState

Terminal state being recorded: complete, pruned, or failed.

values IReadOnlyList<double>

Resulting trial values in objective order; non-null for a successful trial.

BeforeTrial(Study, FrozenTrial)

Trial pre-processing: called during trial initialization, just before the InferRelativeSearchSpace(Study, FrozenTrial) call.

public override void BeforeTrial(Study study, FrozenTrial trial)

Parameters

study Study

Study that owns the newly initialized trial.

trial FrozenTrial

New trial snapshot available before search-space inference.

InferRelativeSearchSpace(Study, FrozenTrial)

Infer the search space that will be used by relative sampling in the target trial.

This method is called right before SampleRelative(Study, FrozenTrial, OrderedDict<string, BaseDistribution>), and the search space returned by this method is passed to it. The parameters not contained in the search space will be sampled by using SampleIndependent(Study, FrozenTrial, string, BaseDistribution).

public override OrderedDict<string, BaseDistribution> InferRelativeSearchSpace(Study study, FrozenTrial trial)

Parameters

study Study

Study supplying trial history, directions, and storage.

trial FrozenTrial

Current trial snapshot; copy it before modifying its contents.

Returns

OrderedDict<string, BaseDistribution>

Parameter distributions eligible for joint sampling, in sampler-defined order.

ReseedRng()

Reseed sampler's random number generator.

Parallel optimization calls this hook before a worker's sequential trial loop. Concrete samplers refresh their random streams so workers do not deliberately reuse a seeded sequence. OptunaSharp workers share the study's sampler instance; this hook does not clone it or make concurrent sampling reproducible.

public override void ReseedRng()

SampleIndependent(Study, FrozenTrial, string, BaseDistribution)

Sample a parameter for a given distribution.

This method is called only for the parameters not contained in the search space returned by SampleRelative(Study, FrozenTrial, OrderedDict<string, BaseDistribution>). This method is suitable for sampling algorithms that do not use relationship between parameters such as random sampling and TPE.

public override object? SampleIndependent(Study study, FrozenTrial trial, string paramName, BaseDistribution paramDistribution)

Parameters

study Study

Study supplying trial history, directions, and storage.

trial FrozenTrial

Current trial snapshot; copy it before modifying its contents.

paramName string

Name of the parameter being suggested.

paramDistribution BaseDistribution

Distribution defining the parameter's domain and scale.

Returns

object

A value in the distribution's external representation, including null categorical choices.

Exceptions

ValueError

The study has multiple objectives, but this sampler requires a scalar objective.

SampleRelative(Study, FrozenTrial, OrderedDict<string, BaseDistribution>)

Sample parameters in a given search space.

The trial invokes this method lazily when it first needs relative parameters and caches the result for its remaining suggestions. This method is suitable for sampling algorithms that use relationship between parameters such as Gaussian Process and CMA-ES.

public override OrderedDict<string, object?> SampleRelative(Study study, FrozenTrial trial, OrderedDict<string, BaseDistribution> searchSpace)

Parameters

study Study

Study supplying trial history, directions, and storage.

trial FrozenTrial

Current trial snapshot; copy it before modifying its contents.

searchSpace OrderedDict<string, BaseDistribution>

Distributions returned by InferRelativeSearchSpace(Study, FrozenTrial).

Returns

OrderedDict<string, object>

Jointly sampled external parameter values; omitted parameters use independent sampling.

Exceptions

ValueError

The study has multiple objectives, but this sampler requires a scalar objective.

See Also

Official Optuna CmaEsSampler reference.
Edit this page