Class TurboSampler

Namespace

OptunaSharp.Samplers

Assembly

OptunaSharp.dll

A sampler using TuRBO — trust-region Bayesian optimization (Eriksson et al. 2019). It fits a local Matérn-5/2 GP to the observations of the current local run, keeps a trust region (a length-scale-weighted hyperrectangle) around the incumbent, and proposes the next point by Thompson sampling over a Sobol perturbation set inside that region. The region expands after consecutive improvements, shrinks after stalls, and restarts once it collapses — the mechanism that lets TuRBO keep making progress in higher dimension where a single global GP (see GPSampler) over-explores.

TuRBO-1 vs TuRBO-m. With the default numTrustRegions = 1 this is TuRBO-1, a single trust region restarted sequentially. With numTrustRegions = m > 1 it is TuRBO-m: m trust regions run simultaneously, each an independent TuRBO-1 with its own local GP, incumbent, and resize counters, and an implicit multi-armed bandit (a Thompson sample drawn from every region, global best wins) allocates each evaluation to the most promising region. TuRBO-m concentrates the budget on the best of several basins (§3.7) but divides that budget across the regions, so it is not uniformly better — it wins on multimodal/deceptive problems and can tie or slightly lose on easy ones; the paper accordingly reports TuRBO-1 and TuRBO-5/-20 by regime.

Beyond-core, convergence-validated. Optuna itself has no TuRBO. OptunaHub has one, by the Optuna team (samplers/turbo, TuRBOSampler, since 2025-09), which this class does not follow: it ports the paper (Eriksson et al. 2019), so a study moved over from the hub package gets other defaults and other trials (map below; QA-70, owner's decision of 2026-11-12 to keep this port). So — like LmCmaSampler and DifferentialEvolutionSampler — it is validated by convergence rather than parity. GP fitting + Thompson sampling ride the same native GP stack as GPSampler (hand-derived Matérn-5/2 marginal-likelihood gradient, managed L-BFGS-B, OpenBLAS Cholesky), so results are tolerance-reproducible on a platform, not bit-exact vs BoTorch.

OptunaHub's TuRBOSampler and this class (registry pinned in refs/optunahub-registry):

OptunaHub TuRBOSampler	This class
n_trust_region=5	numTrustRegions = 1 (TuRBO-1; pass 5 for TuRBO-5)
n_startup_trials=4	nInit: 2·d Sobol points per (re)start
failure_tolerance=5	failureTolerance: the paper's d
success_tolerance=3, init_length=0.8, max_length=1.6, min_length=0.5**7	successTolerance = 3, lengthInit = 0.8, lengthMax = 1.6, lengthMin = 1/128: the same values
seed, independent_sampler, deterministic_objective, warn_independent_sampling	the same names in C# casing; plus numCandidates, successMargin, minimumNoise

Continuous / integer only. The trust-region box is a continuous construct, so the relative search space is Float/Int only; categorical and single-valued parameters are delegated to the independent sampler, exactly as in CmaEsSampler. Dynamic search spaces fall back to independent sampling with a warning. Single-objective only.

State is reconstructed, not persisted (with one small exception for TuRBO-m). TuRBO-1 stores no optimizer blob: the trust-region length, resize counters, and restart boundaries are recomputed each trial by a deterministic fold over the completed-trial objective sequence (ReconstructState(double[], int)), so a study resumes from storage with no serialized state. TuRBO-m adds the minimum needed to fold each region: because a trial's region is the seeded competition winner (a function of which prior trials landed in which region, not of the values alone), m > 1 persists one small per-trial region label (system attr "turbo:tr"); each region then folds exactly as TuRBO-1 over its labeled subsequence.

public sealed class TurboSampler : BaseSampler

Inheritance

object

BaseSampler

TurboSampler

Inherited Members

BaseSampler.ToString()

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.ReferenceEquals(object, object)

Examples

// TuRBO-1 (single trust region, the default).
var study = Optuna.CreateStudy(sampler: new TurboSampler(seed: 42));
study.Optimize(t =>
{
    double s = 0;
    for (int i = 0; i < 10; i++) { double x = t.SuggestFloat($"x{i}", -5, 5); s += x * x; }
    return s;
}, nTrials: 200);

// TuRBO-5 (five simultaneous trust regions) for a multimodal objective with several basins.
var multi = Optuna.CreateStudy(sampler: new TurboSampler(seed: 42, numTrustRegions: 5));
multi.Optimize(t =>
{
    double s = 0;
    for (int i = 0; i < 10; i++) { double x = t.SuggestFloat($"x{i}", -5, 5); s += x * x - 10 * Math.Cos(2 * Math.PI * x); }
    return s + 100;
}, nTrials: 400);

Constructors

TurboSampler(BaseSampler?, bool, int?, int?, int?, double, double, double, int, int?, double, double?, bool, int)

Configures TuRBO's trust-region schedule, local-GP fit, region count, and independent startup/fallback.

public TurboSampler(BaseSampler? independentSampler = null, bool warnIndependentSampling = true, int? seed = null, int? nInit = null, int? numCandidates = null, double lengthInit = 0.8, double lengthMin = 0.0078125, double lengthMax = 1.6, int successTolerance = 3, int? failureTolerance = null, double successMargin = 0, double? minimumNoise = null, bool deterministicObjective = false, int numTrustRegions = 1)

Parameters

independentSampler BaseSampler

Sampler for parameters outside the relative search space, and for the per-restart initialization when it is not Sobol (default RandomSampler).

warnIndependentSampling bool

Emit a warning when a parameter is sampled independently.

seed int?

Random seed; a fixed seed reproduces the trajectory on a platform.

nInit int?

Per-restart initialization trials (Sobol space-filling) before the local GP takes over; null uses 2·d.

numCandidates int?

Thompson-sampling candidate-set size per trial; null uses the paper's min(100·d, 5000) (§D). Cost footgun: this exact managed implementation draws the joint posterior via a dense Cholesky (cost O(numCandidates³) per trial), whereas the paper leans on BoTorch's scalable posterior sampler — so at high d the paper-faithful default is genuinely heavy (a 5000-wide Cholesky per trial). Lower it (e.g. a few hundred) to trade discretization fidelity for speed when the per-trial overhead dominates.

lengthInit double

Initial trust-region base side length on a fresh restart (default 0.8).

lengthMin double

Restart floor: when the base length falls below this the region is discarded and re-initialized (default 2⁻⁷ ≈ 0.0078).

lengthMax double

Cap on the base side length under expansion (default 1.6).

successTolerance int

Consecutive improvements required to double the region (default 3, the paper supplement value; the BoTorch tutorial uses 10).

failureTolerance int?

Consecutive stalls required to halve the region; null uses the paper's τ_fail = ⌈d/q⌉ = d (batch q = 1; §D). The authors' reference code and the BoTorch tutorial instead use ⌈max(4/q, d/q)⌉ = max(4, d) as a low-dimension guard — identical for d ≥ 4 (the paper's whole experimental regime) and different only for d ∈ {1, 2, 3}. Pass Math.Max(4, d) to reproduce that reference-code behavior.

successMargin double

Relative margin a candidate must clear over the incumbent to count as a success: score > best + successMargin·|best|. Default 0.0 is the paper's strict improvement (§D); the BoTorch tutorial uses 1e-3 to ignore negligible gains — pass that to recover BoTorch behavior.

minimumNoise double?

Lower floor on the fitted GP observation-noise variance and the effective lower bound of the §C noise box; null uses the paper's σ² ≥ 0.0005 (§C). Must stay below the §C upper bound 0.1 for a well-posed noise box.

deterministicObjective bool

Treat the objective as noise-free: fix the GP noise at the floor and center the trust region on the best observation (paper §2). When false (the default, noisy case) the center is the training point with the best posterior mean instead.

numTrustRegions int

Number of simultaneous trust regions m (paper §2). Default 1 is TuRBO-1 (single region), a provably byte-identical path — the m > 1 multi-region bandit code never runs at m = 1. Pass 5 for the paper's common "TuRBO-5". Consequences of m > 1: the evaluation budget is divided across the regions (each pays a 2·d init tax and gets fewer exploitation steps), so TuRBO-m is not uniformly better — it wins on multimodal/deceptive problems with several basins (§3.7) and can tie or slightly lose on easy/unimodal ones (which is why the paper reports TuRBO-1 and TuRBO-5/-20 by regime). And because a trial's region is the seeded competition winner (not a function of the values alone), m > 1 persists one small per-trial region label to storage (system attr "turbo:tr"); a trial whose relative search space is empty (all-categorical) carries no label and never joins a region.

Exceptions

ArgumentException

A length bound is non-positive/ill-ordered, or a count is below one.

Methods

AfterTrial(Study, FrozenTrial, TrialState, IReadOnlyList<double>?)

Trial post-processing: called after the objective function returns and right before the trial is finished and its state is stored.

public override void AfterTrial(Study study, FrozenTrial trial, TrialState state, IReadOnlyList<double>? values)

Parameters

study Study

Study that owns the trial being finished.

trial FrozenTrial

Trial snapshot before its terminal state is written to storage.

state TrialState

Terminal state being recorded: complete, pruned, or failed.

values IReadOnlyList<double>

Resulting trial values in objective order; non-null for a successful trial.

BeforeTrial(Study, FrozenTrial)

Trial pre-processing: called during trial initialization, just before the InferRelativeSearchSpace(Study, FrozenTrial) call.

public override void BeforeTrial(Study study, FrozenTrial trial)

Parameters

study Study

Study that owns the newly initialized trial.

trial FrozenTrial

New trial snapshot available before search-space inference.

InferRelativeSearchSpace(Study, FrozenTrial)

Infer the search space that will be used by relative sampling in the target trial.

This method is called right before SampleRelative(Study, FrozenTrial, OrderedDict<string, BaseDistribution>), and the search space returned by this method is passed to it. The parameters not contained in the search space will be sampled by using SampleIndependent(Study, FrozenTrial, string, BaseDistribution).

public override OrderedDict<string, BaseDistribution> InferRelativeSearchSpace(Study study, FrozenTrial trial)

Parameters

study Study

Study supplying trial history, directions, and storage.

trial FrozenTrial

Current trial snapshot; copy it before modifying its contents.

Returns

OrderedDict<string, BaseDistribution>

Parameter distributions eligible for joint sampling, in sampler-defined order.

ReseedRng()

Reseed sampler's random number generator.

Parallel optimization calls this hook before a worker's sequential trial loop. Concrete samplers refresh their random streams so workers do not deliberately reuse a seeded sequence. OptunaSharp workers share the study's sampler instance; this hook does not clone it or make concurrent sampling reproducible.

public override void ReseedRng()

SampleIndependent(Study, FrozenTrial, string, BaseDistribution)

Sample a parameter for a given distribution.

This method is called only for the parameters not contained in the search space returned by SampleRelative(Study, FrozenTrial, OrderedDict<string, BaseDistribution>). This method is suitable for sampling algorithms that do not use relationship between parameters such as random sampling and TPE.

public override object? SampleIndependent(Study study, FrozenTrial trial, string paramName, BaseDistribution paramDistribution)

Parameters

study Study

Study supplying trial history, directions, and storage.

trial FrozenTrial

Current trial snapshot; copy it before modifying its contents.

paramName string

Name of the parameter being suggested.

paramDistribution BaseDistribution

Distribution defining the parameter's domain and scale.

Returns

object

A value in the distribution's external representation, including null categorical choices.

Exceptions

ValueError

The study has multiple objectives, but TuRBO-1 requires a scalar objective.

SampleRelative(Study, FrozenTrial, OrderedDict<string, BaseDistribution>)

Sample parameters in a given search space.

The trial invokes this method lazily when it first needs relative parameters and caches the result for its remaining suggestions. This method is suitable for sampling algorithms that use relationship between parameters such as Gaussian Process and CMA-ES.

public override OrderedDict<string, object?> SampleRelative(Study study, FrozenTrial trial, OrderedDict<string, BaseDistribution> searchSpace)

Parameters

study Study

Study supplying trial history, directions, and storage.

trial FrozenTrial

Current trial snapshot; copy it before modifying its contents.

searchSpace OrderedDict<string, BaseDistribution>

Distributions returned by InferRelativeSearchSpace(Study, FrozenTrial).

Returns

OrderedDict<string, object>

Jointly sampled external parameter values; omitted parameters use independent sampling.

Exceptions

ValueError

The study has multiple objectives, but TuRBO-1 requires a scalar objective.