Table of Contents

Class SaasboSampler

Namespace
OptunaSharp.Samplers
Assembly
OptunaSharp.dll

A sampler using SAASBO — high-dimensional Bayesian optimization with sparse axis-aligned subspaces (Eriksson & Jankowiak 2021). Each trial it fits an RBF ARD Gaussian process under the SAAS prior — a half-Cauchy sparsity prior on the per-dimension inverse-squared length scales that "turns off" most dimensions unless the data demands otherwise (automatic relevance determination) — and proposes the next point by maximizing the expected improvement averaged over posterior hyperparameter samples. This is the paper's answer to the curse of dimensionality: work directly in the full space, but let a sparsity-inducing function prior keep the surrogate parsimonious enough to infer from a few hundred queries.

Inference. The default is the No-U-Turn Sampler (Nuts, the paper's "gold standard"), which averages EI over a bag of L posterior draws; Map is a faster point-estimate alternative (Adam over a small shrinkage grid + leave-one-out selection). The single prior hyperparameter is α (global shrinkage, default 0.1); every other knob is purely computational.

Cost footgun. SAASBO is deliberately expensive: the paper's default NUTS budget (512 warmup + 256 samples, thinned to 16) runs Hamiltonian Monte Carlo over an O(N³D) gradient per leapfrog step, and the default EI maximization screens Q = 5000 Sobol candidates before three L-BFGS-B polishes — appropriate when the objective is expensive to evaluate (the paper targets a few hundred queries) but heavy otherwise. Lower numWarmup/numSamples/numCandidates (or use Map) to trade fidelity for speed. Best for N ≲ 500 observations.

Continuous / integer only. The SAAS GP models a continuous [0, 1]^D box, so the relative search space is Float/Int only; categorical and single-valued parameters are delegated to the independent sampler, exactly as in TurboSampler / CmaEsSampler. Dynamic search spaces fall back to independent sampling with a warning. Single-objective only.

State is reconstructed, not persisted. Like TurboSampler, SAASBO stores no optimizer blob: each trial re-fits the GP to the completed observations, so a study resumes from storage with no serialized state. The one thing it writes is the Sobol row each initialization trial took (the system attr "saasbo:init"), so trials asked at the same time (Optimize(nJobs)) take distinct rows instead of all receiving the row the completed count names.

Conformance to the paper (audited; see SaasboPaperClaimsTests). The RBF kernel (Eqn 1), closed-form posterior (Eqns 4-5), SAAS prior (Eqn 8) with the reparameterization (Eqn 12), the EI acquisition and its sample-averaging (Eqns 6-7, 10), the NUTS budget (§A.1), the MAP recipe (§A.2), and the EI-maximization strategy (§B) are all implemented as stated; the empirical claims of §5.2 (relevant-subspace identification, α controlling shrinkage) and §4.1/§5.7 (adaptive sparsity — more data turns on more dimensions) are asserted on seeded chains. Four deliberate, documented scope choices differ from the paper's full generality, none of them a correctness gap: (1) the RBF kernel only — the paper's default — not the optional Matérn-5/2; (2) a fixed tiny noise (the paper's noise-free primary setting), so the noisy-objective extension (a LN(0,10²) prior on σ²) is not modeled; (3) standardized targets (a numerical convention shared with the reference implementation and the rest of the GP stack); and (4) SAASBO-MAP maximizes the density in the constrained parameter (the paper's literal Eqn 13, "maximize U_s w.r.t. ψ"), not the unconstrained-space mode. The MAP shrinkage grid follows the authoritative §A.2 (S=4, {1,1e-1,1e-2,1e-3}), resolving the main text's smaller §4.2.2 list. Parity is behavioral (HMC is stochastic), not byte-exact vs NumPyro/BoTorch — the TuRBO/REMBO validation tier.

public sealed class SaasboSampler : BaseSampler
Inheritance
SaasboSampler
Inherited Members

Examples

// A 2-D objective embedded in 20 dimensions — SAASBO discovers the two relevant axes.
var study = Optuna.CreateStudy(sampler: new SaasboSampler(
    seed: 0, nStartupTrials: 8,
    // light budget for a fast demo (paper defaults are far heavier):
    numWarmup: 64, numSamples: 64, thinning: 8, numCandidates: 256));
study.Optimize(t =>
{
    double a = t.SuggestFloat("x0", -5, 10), b = t.SuggestFloat("x1", 0, 15);
    for (int i = 2; i < 20; i++) t.SuggestFloat($"x{i}", 0, 1);   // irrelevant dimensions
    return (b - 5.1 / (4 * Math.PI * Math.PI) * a * a + 5 / Math.PI * a - 6) * 2;  // Branin-like
}, nTrials: 40);

Constructors

SaasboSampler(BaseSampler?, bool, int?, int, SaasboInferenceMethod, double, int, int, int, int, double, int, int, int, int, double, double)

Configures SAASBO's Sobol startup, SAAS-GP inference, and EI maximization.

public SaasboSampler(BaseSampler? independentSampler = null, bool warnIndependentSampling = true, int? seed = null, int nStartupTrials = 10, SaasboInferenceMethod inferenceMethod = SaasboInferenceMethod.Nuts, double alpha = 0.1, int numWarmup = 512, int numSamples = 256, int thinning = 16, int maxTreeDepth = 6, double targetAccept = 0.8, int mapSteps = 1500, int numCandidates = 5000, int numRestarts = 3, int maxLbfgsEvals = 100, double noiseVariance = 1E-06, double kernelVarianceLogStd = 10)

Parameters

independentSampler BaseSampler

Sampler for parameters outside the relative search space and for the Sobol-init fallback (default RandomSampler).

warnIndependentSampling bool

Emit a warning when a parameter is sampled independently.

seed int?

Random seed; a fixed seed reproduces the trajectory on a platform.

nStartupTrials int

Sobol initialization trials before the SAAS GP takes over (paper's m; default 10, the paper's Branin/Rosenbrock value). Must be at least 1.

inferenceMethod SaasboInferenceMethod

NUTS (paper default) or MAP.

alpha double

The SAAS global-shrinkage hyperparameter α > 0 (NUTS only; MAP ignores it). Smaller is more conservative (prefers smaller subspaces); the paper's default is 0.1.

numWarmup int

NUTS warmup iterations (paper: 512). Ignored for MAP.

numSamples int

NUTS post-warmup iterations before thinning (paper: 256). Ignored for MAP.

thinning int

NUTS thinning factor — every thinning-th post-warmup draw is retained (paper: 16, giving L = 16 samples). Ignored for MAP.

maxTreeDepth int

NUTS maximum tree depth (paper: 6). Ignored for MAP.

targetAccept double

NUTS dual-averaging target acceptance rate (0.8). Ignored for MAP.

mapSteps int

MAP Adam iterations per shrinkage value (paper: 1500). Ignored for NUTS.

numCandidates int

EI-maximization Sobol screen size Q (paper: 5000).

numRestarts int

EI-maximization L-BFGS-B restarts K (paper: 3).

maxLbfgsEvals int

EI-maximization per-restart L-BFGS-B evaluation cap (paper: 100).

noiseVariance double

Fixed GP observation-noise variance (paper: 1e-6, noise-free). Must be positive.

kernelVarianceLogStd double

Std of the log-Normal prior on the kernel variance σ²_k (paper: 10).

Exceptions

ArgumentException

A count is below one or a positive-valued knob is non-positive.

Methods

AfterTrial(Study, FrozenTrial, TrialState, IReadOnlyList<double>?)

Trial post-processing: called after the objective function returns and right before the trial is finished and its state is stored.

public override void AfterTrial(Study study, FrozenTrial trial, TrialState state, IReadOnlyList<double>? values)

Parameters

study Study

Study that owns the trial being finished.

trial FrozenTrial

Trial snapshot before its terminal state is written to storage.

state TrialState

Terminal state being recorded: complete, pruned, or failed.

values IReadOnlyList<double>

Resulting trial values in objective order; non-null for a successful trial.

BeforeTrial(Study, FrozenTrial)

Trial pre-processing: called during trial initialization, just before the InferRelativeSearchSpace(Study, FrozenTrial) call.

public override void BeforeTrial(Study study, FrozenTrial trial)

Parameters

study Study

Study that owns the newly initialized trial.

trial FrozenTrial

New trial snapshot available before search-space inference.

InferRelativeSearchSpace(Study, FrozenTrial)

Infer the search space that will be used by relative sampling in the target trial.

This method is called right before SampleRelative(Study, FrozenTrial, OrderedDict<string, BaseDistribution>), and the search space returned by this method is passed to it. The parameters not contained in the search space will be sampled by using SampleIndependent(Study, FrozenTrial, string, BaseDistribution).

public override OrderedDict<string, BaseDistribution> InferRelativeSearchSpace(Study study, FrozenTrial trial)

Parameters

study Study

Study supplying trial history, directions, and storage.

trial FrozenTrial

Current trial snapshot; copy it before modifying its contents.

Returns

OrderedDict<string, BaseDistribution>

Parameter distributions eligible for joint sampling, in sampler-defined order.

ReseedRng()

Reseed sampler's random number generator.

Parallel optimization calls this hook before a worker's sequential trial loop. Concrete samplers refresh their random streams so workers do not deliberately reuse a seeded sequence. OptunaSharp workers share the study's sampler instance; this hook does not clone it or make concurrent sampling reproducible.

public override void ReseedRng()

SampleIndependent(Study, FrozenTrial, string, BaseDistribution)

Sample a parameter for a given distribution.

This method is called only for the parameters not contained in the search space returned by SampleRelative(Study, FrozenTrial, OrderedDict<string, BaseDistribution>). This method is suitable for sampling algorithms that do not use relationship between parameters such as random sampling and TPE.

public override object? SampleIndependent(Study study, FrozenTrial trial, string paramName, BaseDistribution paramDistribution)

Parameters

study Study

Study supplying trial history, directions, and storage.

trial FrozenTrial

Current trial snapshot; copy it before modifying its contents.

paramName string

Name of the parameter being suggested.

paramDistribution BaseDistribution

Distribution defining the parameter's domain and scale.

Returns

object

A value in the distribution's external representation, including null categorical choices.

Exceptions

ValueError

The study has multiple objectives, but SAASBO requires a scalar objective.

SampleRelative(Study, FrozenTrial, OrderedDict<string, BaseDistribution>)

Sample parameters in a given search space.

The trial invokes this method lazily when it first needs relative parameters and caches the result for its remaining suggestions. This method is suitable for sampling algorithms that use relationship between parameters such as Gaussian Process and CMA-ES.

public override OrderedDict<string, object?> SampleRelative(Study study, FrozenTrial trial, OrderedDict<string, BaseDistribution> searchSpace)

Parameters

study Study

Study supplying trial history, directions, and storage.

trial FrozenTrial

Current trial snapshot; copy it before modifying its contents.

searchSpace OrderedDict<string, BaseDistribution>

Distributions returned by InferRelativeSearchSpace(Study, FrozenTrial).

Returns

OrderedDict<string, object>

Jointly sampled external parameter values; omitted parameters use independent sampling.

Exceptions

ValueError

The study has multiple objectives, but SAASBO requires a scalar objective.

Edit this page