Table of Contents

Class RandomEmbeddingSampler

Namespace
OptunaSharp.Samplers
Assembly
OptunaSharp.dll

A sampler implementing REMBO — Random EMbedding Bayesian Optimization (Wang et al. 2016). It draws a fixed random Gaussian matrix A ∈ ℝ^{D×d}, runs the paper's inner Bayesian optimizer (by default a squared-exponential GP with expected improvement maximized by DIRECT + CMA-ES and the Algorithm-3 adaptive length scale) in the small box Y = [−√d, √d]^d, and maps each latent point y to the study's real parameters through x = p_X(A·y) (a box-constrained projection). This lets BO scale to a search space of (in principle) enormous dimension D as long as the objective's effective dimensionality d_e is low — the invariance the paper proves and demonstrates up to a billion dimensions.

The one knob that matters: nEmbeddingDims (d). It must be at least the objective's (guessed) effective dimensionality for the embedding to contain the optimum; larger d raises the chance the box captures the optimum but slows the inner optimizer (Section 3.1). There is no universal default — it reflects a belief about the problem — so it is required.

Interleaved runs (nEmbeddings, k). A single embedding's box misses the optimum with some probability δ; running k independent embeddings drops that to δ^k (Section 3.1). With k > 1 trials are assigned to embeddings round-robin, each embedding keeping its own latent history.

How it fits the Optuna protocol. Every real parameter is produced through SampleIndependent(Study, FrozenTrial, string, BaseDistribution) from a single latent point sampled once per trial — so all real parameters of a trial come from one coherent y, and even the very first trial is embedded (its observation feeds the inner optimizer). By default the latent point is chosen by REMBO's paper-faithful optimizer — a squared-exponential GP with expected improvement maximized by DIRECT + CMA-ES and the Algorithm-3 adaptive length scale — fit on the finished trials' latent points; an overridden inner sampler is driven over a private in-memory shadow study of them instead. Either way the latent point is persisted in the real trial's system attributes so the run can be rebuilt after resumption. Float and integer parameters are embedded (integers/steps/log honored via the standard de-normalization); categorical and single-valued parameters are delegated to an independent sampler, as in CmaEsSampler / TurboSampler. Single-objective only.

Beyond-core, convergence-validated. REMBO has no Optuna counterpart and its inner GP is stochastic, so — like TurboSampler and LmCmaSampler — it is validated by convergence (embedding recovery on a low-effective-dimension objective) and same-seed determinism under sequential optimization, not by byte-parity.

public sealed class RandomEmbeddingSampler : BaseSampler
Inheritance
RandomEmbeddingSampler
Inherited Members

Examples

// A 2-effective-dimension objective hidden inside 25 real parameters: REMBO with d = 4 finds it.
var study = Optuna.CreateStudy(sampler: new RandomEmbeddingSampler(nEmbeddingDims: 4, seed: 42));
study.Optimize(t =>
{
    double sum = 0;
    for (int i = 0; i < 25; i++)
    {
        double x = t.SuggestFloat($"x{i}", -5, 5);
        if (i < 2) sum += (x - 2) * (x - 2);   // only x0, x1 matter
    }
    return sum;
}, nTrials: 200);

Constructors

RandomEmbeddingSampler(int, BaseSampler?, int, RandomEmbeddingKernel, BaseSampler?, bool, int, int?)

Configures the random embedding, its inner Bayesian optimizer (low- or high-dimensional kernel), and the independent fallback.

public RandomEmbeddingSampler(int nEmbeddingDims, BaseSampler? innerSampler = null, int nEmbeddings = 1, RandomEmbeddingKernel kernel = RandomEmbeddingKernel.LowDimensional, BaseSampler? independentSampler = null, bool warnIndependentSampling = true, int nStartupTrials = 10, int? seed = null)

Parameters

nEmbeddingDims int

The internal embedding dimension d the inner optimizer searches over — the core REMBO knob. Set it to at least the guessed effective dimensionality of the objective; larger values raise the chance the box captures the optimum but slow convergence.

innerSampler BaseSampler

Optional override for the inner Bayesian optimizer. null (the default) runs REMBO's paper-faithful optimizer directly (a squared-exponential GP with expected improvement maximized by DIRECT + CMA-ES and the Algorithm-3 adaptive length scale). Supplying one instead drives it over a private shadow study of the latent points — a beyond-paper generalization for bringing another history-reconstructing sampler (e.g. TPESampler); samplers that persist an optimizer blob across trials (CMA-ES / ES) are not intended here, and the kernel choice does not apply to an overridden sampler.

nEmbeddings int

The number of interleaved independent embeddings k (Section 3.1), default 1; trials are assigned to embeddings round-robin.

kernel RandomEmbeddingKernel

Which REMBO kernel (§3.2) the paper-faithful optimizer uses. The default LowDimensional models the objective over the latent y; HighDimensional fits the GP in the projected real parameter space (more reliable for a single embedding, and it also embeds categorical parameters). Applies only to the default optimizer, not to an overridden innerSampler.

independentSampler BaseSampler

Sampler for parameters REMBO does not embed — single-valued, and (under the low-dimensional kernel) categorical; null uses RandomSampler.

warnIndependentSampling bool

Emit a warning when a parameter is sampled independently rather than embedded.

nStartupTrials int

Completed trials of an embedding required before its GP takes over from uniform-random latent sampling (default 10). Used by the paper-faithful default optimizer.

seed int?

Random seed anchoring the embedding matrix and the default inner/independent samplers; a fixed seed reproduces the trajectory under sequential (single-worker) optimization.

Exceptions

ArgumentException

nEmbeddingDims or nEmbeddings is below one.

Methods

AfterTrial(Study, FrozenTrial, TrialState, IReadOnlyList<double>?)

Trial post-processing: called after the objective function returns and right before the trial is finished and its state is stored.

public override void AfterTrial(Study study, FrozenTrial trial, TrialState state, IReadOnlyList<double>? values)

Parameters

study Study

Study that owns the trial being finished.

trial FrozenTrial

Trial snapshot before its terminal state is written to storage.

state TrialState

Terminal state being recorded: complete, pruned, or failed.

values IReadOnlyList<double>

Resulting trial values in objective order; non-null for a successful trial.

BeforeTrial(Study, FrozenTrial)

Trial pre-processing: called during trial initialization, just before the InferRelativeSearchSpace(Study, FrozenTrial) call.

public override void BeforeTrial(Study study, FrozenTrial trial)

Parameters

study Study

Study that owns the newly initialized trial.

trial FrozenTrial

New trial snapshot available before search-space inference.

InferRelativeSearchSpace(Study, FrozenTrial)

Infer the search space that will be used by relative sampling in the target trial.

This method is called right before SampleRelative(Study, FrozenTrial, OrderedDict<string, BaseDistribution>), and the search space returned by this method is passed to it. The parameters not contained in the search space will be sampled by using SampleIndependent(Study, FrozenTrial, string, BaseDistribution).

public override OrderedDict<string, BaseDistribution> InferRelativeSearchSpace(Study study, FrozenTrial trial)

Parameters

study Study

Study supplying trial history, directions, and storage.

trial FrozenTrial

Current trial snapshot; copy it before modifying its contents.

Returns

OrderedDict<string, BaseDistribution>

Parameter distributions eligible for joint sampling, in sampler-defined order.

Remarks

REMBO produces every parameter through SampleIndependent(Study, FrozenTrial, string, BaseDistribution) from one shared latent point, so it declares no relative search space (like RandomSampler).

ReseedRng()

Reseed sampler's random number generator.

Parallel optimization calls this hook before a worker's sequential trial loop. Concrete samplers refresh their random streams so workers do not deliberately reuse a seeded sequence. OptunaSharp workers share the study's sampler instance; this hook does not clone it or make concurrent sampling reproducible.

public override void ReseedRng()

SampleIndependent(Study, FrozenTrial, string, BaseDistribution)

Sample a parameter for a given distribution.

This method is called only for the parameters not contained in the search space returned by SampleRelative(Study, FrozenTrial, OrderedDict<string, BaseDistribution>). This method is suitable for sampling algorithms that do not use relationship between parameters such as random sampling and TPE.

public override object? SampleIndependent(Study study, FrozenTrial trial, string paramName, BaseDistribution paramDistribution)

Parameters

study Study

Study supplying trial history, directions, and storage.

trial FrozenTrial

Current trial snapshot; copy it before modifying its contents.

paramName string

Name of the parameter being suggested.

paramDistribution BaseDistribution

Distribution defining the parameter's domain and scale.

Returns

object

A value in the distribution's external representation, including null categorical choices.

Exceptions

ValueError

The study has multiple objectives, but REMBO requires a scalar objective.

SampleRelative(Study, FrozenTrial, OrderedDict<string, BaseDistribution>)

Sample parameters in a given search space.

The trial invokes this method lazily when it first needs relative parameters and caches the result for its remaining suggestions. This method is suitable for sampling algorithms that use relationship between parameters such as Gaussian Process and CMA-ES.

public override OrderedDict<string, object?> SampleRelative(Study study, FrozenTrial trial, OrderedDict<string, BaseDistribution> searchSpace)

Parameters

study Study

Study supplying trial history, directions, and storage.

trial FrozenTrial

Current trial snapshot; copy it before modifying its contents.

searchSpace OrderedDict<string, BaseDistribution>

Distributions returned by InferRelativeSearchSpace(Study, FrozenTrial).

Returns

OrderedDict<string, object>

Jointly sampled external parameter values; omitted parameters use independent sampling.

Edit this page