Table of Contents

Class VkdCmaSampler

Namespace
OptunaSharp.Samplers
Assembly
OptunaSharp.dll

A sampler driving the OptunaSharp.Samplers.CmaEs.VkdCma optimizer — VkD-CMA-ES (Akimoto & Hansen, GECCO/PPSN 2016), a port of Akimoto's own GaussVkDSampler. VkD-CMA restricts the covariance to a rank-k term plus a diagonal and adapts the diagonal, the low-rank basis, AND the rank k online — cost O(N k² + k³) per generation, the middle point between a diagonal-only model (CmaEsSampler with useSeparableCma) and full CmaEsSampler's O(N²) covariance. It targets high-dimensional problems whose covariance is well-approximated by a few dominant directions, and learns how many directions (k) that is as it runs.

Distinct from CmaEsSampler and LmCmaSampler. Like LmCmaSampler, this is a separate standalone sampler, not a flag on CmaEsSampler, and it shares none of that sampler's with_margin / use_separable_cma / warm-start / restart machinery. VkD-CMA is not part of the Optuna ecosystem (no cmaes class, no Optuna sampler, no OptunaHub package), so — like LmCmaSampler — it is a native, reference-faithful implementation validated by convergence, with no upstream to bit-match. It sits beside LmCmaSampler in the restricted-covariance family (both approximate the covariance; LM-CMA with stored direction vectors, VkD-CMA with an adapting rank-k+diagonal factorization).

Two concessions the per-trial protocol forces (both documented on the optimizer): (1) The optimizer is unconstrained; Optuna needs in-box params, so the sampler clamps each asked candidate to the transformed bounds and the next tell re-derives x from the stored (clamped) params — the optimizer learns from the actually-evaluated points. (2) VkD-CMA adapts its step-size by TPA, which injects a mirrored pair as a generation's first two samples and recovers them by direction; Optuna asks one candidate per trial from freshly-restored state, so the sampler drives the generation through AskCandidate(int), passing the true generation position — positions 1 and 2 emit the exact ±dx directions (with independent lengths, which TPA's direction recovery does not depend on).

Categorical parameters and single-value distributions are handled by the independent sampler, as in CmaEsSampler. The optimizer needs n > 1; a single-dimensional relative space falls back to independent sampling.

public sealed class VkdCmaSampler : BaseSampler
Inheritance
VkdCmaSampler
Inherited Members

Examples

var study = Optuna.CreateStudy(sampler: new VkdCmaSampler(seed: 42));
study.Optimize(t =>
{
    double s = 0;
    for (int i = 0; i < 20; i++) { double x = t.SuggestFloat($"x{i}", -5, 5); s += x * x; }
    return s;
}, nTrials: 200);

Constructors

VkdCmaSampler(int, BaseSampler?, bool, int?, bool, int?, int?, bool)

Configures covariance adaptation with a diagonal-plus-low-rank model of the numerical search space.

public VkdCmaSampler(int nStartupTrials = 1, BaseSampler? independentSampler = null, bool warnIndependentSampling = true, int? seed = null, bool considerPrunedTrials = false, int? popsize = null, int? kInit = null, bool kAdapt = true)

Parameters

nStartupTrials int

Independent sampling is used until this many trials finish.

independentSampler BaseSampler

Sampler for parameters outside the relative search space (default RandomSampler).

warnIndependentSampling bool

Emit a warning when a parameter is sampled independently.

seed int?

Random seed for VkD-CMA.

considerPrunedTrials bool

If true, PRUNED trials are considered for sampling (experimental).

popsize int?

Population size λ (default 4 + ⌊3 ln n⌋).

kInit int?

Initial rank k of the low-rank covariance term (default 0 — starts diagonal and grows).

kAdapt bool

Whether the rank k adapts online (PPSN-2016 model selection); default true.

Methods

AfterTrial(Study, FrozenTrial, TrialState, IReadOnlyList<double>?)

Trial post-processing: called after the objective function returns and right before the trial is finished and its state is stored.

public override void AfterTrial(Study study, FrozenTrial trial, TrialState state, IReadOnlyList<double>? values)

Parameters

study Study

Study that owns the trial being finished.

trial FrozenTrial

Trial snapshot before its terminal state is written to storage.

state TrialState

Terminal state being recorded: complete, pruned, or failed.

values IReadOnlyList<double>

Resulting trial values in objective order; non-null for a successful trial.

BeforeTrial(Study, FrozenTrial)

Trial pre-processing: called during trial initialization, just before the InferRelativeSearchSpace(Study, FrozenTrial) call.

public override void BeforeTrial(Study study, FrozenTrial trial)

Parameters

study Study

Study that owns the newly initialized trial.

trial FrozenTrial

New trial snapshot available before search-space inference.

InferRelativeSearchSpace(Study, FrozenTrial)

Infer the search space that will be used by relative sampling in the target trial.

This method is called right before SampleRelative(Study, FrozenTrial, OrderedDict<string, BaseDistribution>), and the search space returned by this method is passed to it. The parameters not contained in the search space will be sampled by using SampleIndependent(Study, FrozenTrial, string, BaseDistribution).

public override OrderedDict<string, BaseDistribution> InferRelativeSearchSpace(Study study, FrozenTrial trial)

Parameters

study Study

Study supplying trial history, directions, and storage.

trial FrozenTrial

Current trial snapshot; copy it before modifying its contents.

Returns

OrderedDict<string, BaseDistribution>

Parameter distributions eligible for joint sampling, in sampler-defined order.

ReseedRng()

Reseed sampler's random number generator.

Parallel optimization calls this hook before a worker's sequential trial loop. Concrete samplers refresh their random streams so workers do not deliberately reuse a seeded sequence. OptunaSharp workers share the study's sampler instance; this hook does not clone it or make concurrent sampling reproducible.

public override void ReseedRng()

SampleIndependent(Study, FrozenTrial, string, BaseDistribution)

Sample a parameter for a given distribution.

This method is called only for the parameters not contained in the search space returned by SampleRelative(Study, FrozenTrial, OrderedDict<string, BaseDistribution>). This method is suitable for sampling algorithms that do not use relationship between parameters such as random sampling and TPE.

public override object? SampleIndependent(Study study, FrozenTrial trial, string paramName, BaseDistribution paramDistribution)

Parameters

study Study

Study supplying trial history, directions, and storage.

trial FrozenTrial

Current trial snapshot; copy it before modifying its contents.

paramName string

Name of the parameter being suggested.

paramDistribution BaseDistribution

Distribution defining the parameter's domain and scale.

Returns

object

A value in the distribution's external representation, including null categorical choices.

Exceptions

ValueError

The study has multiple objectives, but this sampler requires a scalar objective.

SampleRelative(Study, FrozenTrial, OrderedDict<string, BaseDistribution>)

Sample parameters in a given search space.

The trial invokes this method lazily when it first needs relative parameters and caches the result for its remaining suggestions. This method is suitable for sampling algorithms that use relationship between parameters such as Gaussian Process and CMA-ES.

public override OrderedDict<string, object?> SampleRelative(Study study, FrozenTrial trial, OrderedDict<string, BaseDistribution> searchSpace)

Parameters

study Study

Study supplying trial history, directions, and storage.

trial FrozenTrial

Current trial snapshot; copy it before modifying its contents.

searchSpace OrderedDict<string, BaseDistribution>

Distributions returned by InferRelativeSearchSpace(Study, FrozenTrial).

Returns

OrderedDict<string, object>

Jointly sampled external parameter values; omitted parameters use independent sampling.

Exceptions

ValueError

The study has multiple objectives, but this sampler requires a scalar objective.

Edit this page