Table of Contents

Class LmCmaSampler

Namespace
OptunaSharp.Samplers
Assembly
OptunaSharp.dll

A sampler driving the paper-identical OptunaSharp.Samplers.CmaEs.LmCma optimizer — Limited-Memory CMA-ES (Ilya Loshchilov, "LM-CMA: an Alternative to L-BFGS for Large Scale Black-box Optimization", Evolutionary Computation 2015; arXiv:1511.00221). LM-CMA stores only m ≪ n direction vectors and reconstructs the Cholesky factor on the fly (O(mn) memory and per-sample time, no eigendecomposition), which targets very high-dimensional search spaces where full OptunaSharp.Samplers.CmaEs.Cma's O(n²) covariance is impractical. On low-dimensional / ill-conditioned problems it converges more slowly than full CMA-ES — that is the documented large-scale trade-off, not a defect.

Distinct from CmaEsSampler. This is a separate, standalone sampler, not a flag on CmaEsSampler: LM-CMA shares none of that sampler's with_margin / use_separable_cma / warm-start / restart machinery, and reusing it would force-fit features the paper does not define. LM-CMA is not part of the Optuna ecosystem (no cmaes class, no Optuna sampler, no OptunaHub package), so — unlike the other CMA variants — this is a native paper-faithful implementation validated by convergence, with no upstream to bit-match.

Two paper concessions the per-trial protocol forces (both documented on the optimizer): (1) LM-CMA (the paper) is unconstrained; Optuna needs in-box params, so the sampler clamps each asked candidate to the transformed bounds and the next tell re-derives x from the stored (clamped) params — the optimizer learns from the actually-evaluated points. (2) Optuna asks one candidate per trial from freshly-restored state, which cannot carry the intra-generation mirror pairing (Alg. 7 l.9–10), so the sampler drives the non-mirrored generation via AskCandidate(int), passing the true generation position so SelectSubset's k==1 boost still fires once per generation.

Categorical parameters and single-value distributions are handled by the independent sampler, as in CmaEsSampler. The optimizer needs n > 1; a single-dimensional relative space falls back to independent sampling.

public sealed class LmCmaSampler : BaseSampler
Inheritance
LmCmaSampler
Inherited Members

Examples

var study = Optuna.CreateStudy(sampler: new LmCmaSampler(seed: 42));
study.Optimize(t =>
{
    double s = 0;
    for (int i = 0; i < 20; i++) { double x = t.SuggestFloat($"x{i}", -5, 5); s += x * x; }
    return s;
}, nTrials: 200);

Constructors

LmCmaSampler(int, BaseSampler?, bool, int?, bool, int?, int?)

Configures limited-memory covariance adaptation for a fixed numerical search space.

public LmCmaSampler(int nStartupTrials = 1, BaseSampler? independentSampler = null, bool warnIndependentSampling = true, int? seed = null, bool considerPrunedTrials = false, int? popsize = null, int? nVectors = null)

Parameters

nStartupTrials int

Independent sampling is used until this many trials finish.

independentSampler BaseSampler

Sampler for parameters outside the relative search space (default RandomSampler).

warnIndependentSampling bool

Emit a warning when a parameter is sampled independently.

seed int?

Random seed for LM-CMA.

considerPrunedTrials bool

If true, PRUNED trials are considered for sampling (experimental).

popsize int?

Population size λ (default 4 + ⌊3 ln n⌋, the paper's value).

nVectors int?

Number of stored direction vectors m (default 4 + ⌊3 ln n⌋; the paper notes ≈√n can do better when memory allows).

Methods

AfterTrial(Study, FrozenTrial, TrialState, IReadOnlyList<double>?)

Trial post-processing: called after the objective function returns and right before the trial is finished and its state is stored.

public override void AfterTrial(Study study, FrozenTrial trial, TrialState state, IReadOnlyList<double>? values)

Parameters

study Study

Study that owns the trial being finished.

trial FrozenTrial

Trial snapshot before its terminal state is written to storage.

state TrialState

Terminal state being recorded: complete, pruned, or failed.

values IReadOnlyList<double>

Resulting trial values in objective order; non-null for a successful trial.

BeforeTrial(Study, FrozenTrial)

Trial pre-processing: called during trial initialization, just before the InferRelativeSearchSpace(Study, FrozenTrial) call.

public override void BeforeTrial(Study study, FrozenTrial trial)

Parameters

study Study

Study that owns the newly initialized trial.

trial FrozenTrial

New trial snapshot available before search-space inference.

InferRelativeSearchSpace(Study, FrozenTrial)

Infer the search space that will be used by relative sampling in the target trial.

This method is called right before SampleRelative(Study, FrozenTrial, OrderedDict<string, BaseDistribution>), and the search space returned by this method is passed to it. The parameters not contained in the search space will be sampled by using SampleIndependent(Study, FrozenTrial, string, BaseDistribution).

public override OrderedDict<string, BaseDistribution> InferRelativeSearchSpace(Study study, FrozenTrial trial)

Parameters

study Study

Study supplying trial history, directions, and storage.

trial FrozenTrial

Current trial snapshot; copy it before modifying its contents.

Returns

OrderedDict<string, BaseDistribution>

Parameter distributions eligible for joint sampling, in sampler-defined order.

ReseedRng()

Reseed sampler's random number generator.

Parallel optimization calls this hook before a worker's sequential trial loop. Concrete samplers refresh their random streams so workers do not deliberately reuse a seeded sequence. OptunaSharp workers share the study's sampler instance; this hook does not clone it or make concurrent sampling reproducible.

public override void ReseedRng()

SampleIndependent(Study, FrozenTrial, string, BaseDistribution)

Sample a parameter for a given distribution.

This method is called only for the parameters not contained in the search space returned by SampleRelative(Study, FrozenTrial, OrderedDict<string, BaseDistribution>). This method is suitable for sampling algorithms that do not use relationship between parameters such as random sampling and TPE.

public override object? SampleIndependent(Study study, FrozenTrial trial, string paramName, BaseDistribution paramDistribution)

Parameters

study Study

Study supplying trial history, directions, and storage.

trial FrozenTrial

Current trial snapshot; copy it before modifying its contents.

paramName string

Name of the parameter being suggested.

paramDistribution BaseDistribution

Distribution defining the parameter's domain and scale.

Returns

object

A value in the distribution's external representation, including null categorical choices.

Exceptions

ValueError

The study has multiple objectives, but this sampler requires a scalar objective.

SampleRelative(Study, FrozenTrial, OrderedDict<string, BaseDistribution>)

Sample parameters in a given search space.

The trial invokes this method lazily when it first needs relative parameters and caches the result for its remaining suggestions. This method is suitable for sampling algorithms that use relationship between parameters such as Gaussian Process and CMA-ES.

public override OrderedDict<string, object?> SampleRelative(Study study, FrozenTrial trial, OrderedDict<string, BaseDistribution> searchSpace)

Parameters

study Study

Study supplying trial history, directions, and storage.

trial FrozenTrial

Current trial snapshot; copy it before modifying its contents.

searchSpace OrderedDict<string, BaseDistribution>

Distributions returned by InferRelativeSearchSpace(Study, FrozenTrial).

Returns

OrderedDict<string, object>

Jointly sampled external parameter values; omitted parameters use independent sampling.

Exceptions

ValueError

The study has multiple objectives, but this sampler requires a scalar objective.

Edit this page