Namespace OptunaSharp.Samplers

Classes

AGE-MOEA-2-CPT ("AGE-MOEA-2 with Categorical Parameter Trees") — AGE-MOEA-2's geometry-based environmental selection (AgeMoea2ElitePopulationSelectionStrategy) joined to the conditional-parameter machinery NSGAIIICptSampler introduced, so the define-by-run, branch-defined hyperparameters Rock's gene trees produce are evolved instead of resampled at random. It is the AGE-MOEA-2 sibling of NSGAIIICptSampler and reuses the same CPT seams (NsgaiiiCptSearchSpace, NsgaiiiCptChildGenerationStrategy, ConditionalParams); only the survival rule and the default operators differ.

Why reach for this over NSGAIIICptSampler. Both handle conditional parameters identically. They differ only in survival: NSGA-III niches against a fixed reference-point simplex whose size grows as C(M+2, 3), so past ~5 objectives most of the population is mutually non-dominated and the front stops selecting; AGE-MOEA-2 estimates the front's geometry adaptively and keeps selection pressure in that regime. So this sampler is the one to use when the objective count is (or may grow) high; at 2–4 objectives the two are close and NSGA-III-CPT remains the conservative default. This is a genuine tradeoff, not a strict improvement — validate the swap on the actual objective set before adopting it.

Relationship to AgeMoea2Sampler. Constructed with conditionalParams: null this sampler is behaviorally identical to AgeMoea2Sampler (the CPT search space and child generation both fall through to the shared Optuna-parity fold, and the elite strategy, operators, and RNG draw order are the same), so it is a strict superset. The conditional behavior is the same deliberate, no-Optuna-analog divergence documented on ConditionalParams; AGE-MOEA-2 itself (Panichella, GECCO 2022) is a beyond-core, convergence-validated sampler with no Optuna counterpart, so no ExperimentalWarning is emitted. Unconstrained multi-objective only — AGE-MOEA-2's survival does not model constraints, so unlike NSGAIIICptSampler this class exposes no constraintsFunc.

AgeMoea2Sampler

Multi-objective sampler using AGE-MOEA-2 (Layer 2 of the AGE-MOEA-2 port). The Optuna-idiomatic face of the algorithm — an ask/tell BaseGASampler that reconstructs the population from completed trials exactly like NSGAIISampler, but replaces NSGA-II's rank + crowding-distance elite selection with AGE-MOEA-2's geometry-based environmental selection (AgeMoea2ElitePopulationSelectionStrategy over OptunaSharp.Samplers.AgeMoea.AgeMoea2Survival). Children are generated with SBX + polynomial mutation over Optuna's transformed search space, so the sampler works across Optuna's full parameter space (float / int / categorical). For conditional (branch-defined) parameters use NSGAIIICptSampler (NSGA-III-CPT); this sampler has no such extension.

AGE-MOEA-2 (Panichella, GECCO 2022, doi:10.1145/3512290.3528732) is absent from Optuna core, so — like LM-CMA and JANUS — this is a beyond-core sampler. Scope of the parity guarantee: the byte-perfect, pymoo-faithful AGE-MOEA-2 lives in Layer 1 (OptunaSharp.Samplers.AgeMoea.AgeMoea2Optimizer, generation-by-generation bit-exact vs pymoo.minimize). This Layer-2 sampler is validated behaviorally: it uses Optuna's operators (a different RNG stream than pymoo's), so it is not pymoo-bit-exact — it brings AGE-MOEA-2's survival to Optuna's define-by-run workflow. Unconstrained multi-objective.

BaseGASampler

Base class for Genetic Algorithm (GA) samplers. Port of optuna/samplers/_ga/_base.py: BaseGASampler.

GA samplers generate new trials by mimicking natural selection, using generations and populations to iteratively improve solutions. This base class defines the interface and provides utility methods for managing generations and populations. Parent selection is handled by SelectParent(Study, int), which subclasses implement.

BaseSampler

Base class for samplers. Port of optuna/samplers/_base.py: BaseSampler.

Optuna combines two types of sampling strategies, which are called relative sampling and independent sampling.

The relative sampling determines values of multiple parameters simultaneously so that sampling algorithms can use relationship between parameters (e.g., correlation). Target parameters of the relative sampling are described in a relative search space, which is determined by InferRelativeSearchSpace(Study, FrozenTrial).

The independent sampling determines a value of a single parameter without considering any relationship between parameters. Target parameters of the independent sampling are the parameters not described in the relative search space.

BruteForceSampler

Sampler that performs exhaustive search over the define-by-run search space. Port of optuna/samplers/_brute_force.py: BruteForceSampler (experimental since 3.1.0).

This sampler performs exhaustive search on the defined search space. Each suggestion builds a tree over the trials observed so far — each internal node is a parameter and its children are keyed by the parameter's candidate values (internal repr); a root-to-leaf path is one complete params — then samples an unexplored branch, blending exact-uniform with flat-uniform weighting to prevent starvation of unexplored branches. It stops the study (internally invoking Stop()) once the space is exhausted.

CmaEsSampler

A sampler using cmaes as the backend. Port of optuna/samplers/_cmaes.py: CmaEsSampler driving the OptunaSharp.Samplers.CmaEs.Cma optimizer.

This sampler does not support CategoricalDistribution. If the search space includes categorical parameters they are sampled by the independent sampler.

Optimizer state is Optuna's pickle. Like Optuna, the sampler stores pickle.dumps(optimizer).hex() of a cmaes CMA, SepCMA or CMAwM under the chunked cma:optimizer:{i} (sepcma:, cmawm:) system attrs, written and read by OptunaSharp.Samplers.CmaEs.CmaPickle, so C# and Python workers can share one CMA-ES study in any order. A stored state is byte-identical to the one cmaes 0.13.1 stores for the same optimizer, except the _funhist_values entries cmaes leaves uninitialized (np.empty) until its first generations fill them. States written by OptunaSharp before 2026-10-29 (a cma\n-tagged JSON) are still read; Python cannot read those, so a study gains Python workers once a current C# worker has stored the next state. With with_margin=True the Python workers need cmaes 0.13.1, whose CMAwM state OptunaSharp reads.

Faster than Optuna's structure, same results. Optuna restores the optimizer from its pickle and rescans every trial on every ask, which makes a run quadratic in its trial count. This sampler keeps the restored optimizer for the rest of its generation and folds the history incrementally (CmaTrialHistory), reading an in-memory study through its zero-copy view; seeded trials and every stored byte stay identical to Optuna's (Rustuna plan 13, phase 1; the measured gain is in docs/plans/13-rustuna-derived-performance.md).

DifferentialEvolutionSampler

A sampler driving a textbook-faithful OptunaSharp.Samplers.Differential.De Differential Evolution optimizer (Storn & Price 1997) through the Optuna protocol. DE maintains a population of candidate solutions and, for each target member, forms a trial vector by adding a scaled difference of other members (DE/rand/1 or DE/best/1) and binomially crossing it with the target; the trial replaces the target only if it is no worse (greedy one-to-one selection). It is a robust, derivative-free global optimizer that complements the Gaussian model of CmaEsSampler and the limited-memory LmCmaSampler.

Bootstrapping. DE needs an initial population before it can generate offspring, so the first max(nStartupTrials, populationSize) completed trials are drawn by the independent sampler and become the initial population; DE offspring are proposed from then on.

Two protocol concessions (as LmCmaSampler): (1) DE is unconstrained, so each asked candidate is clamped to the transformed bounds and the next generation's greedy selection re-derives the parent from the stored (clamped) params — the optimizer learns from the actually-evaluated points. (2) The per-trial protocol asks one candidate per trial from freshly-restored state; each trial is stamped with its generation and its target index so the population's greedy selection pairs each offspring with the right parent once the generation completes.

Categorical parameters and single-value distributions are handled by the independent sampler, as in CmaEsSampler.

Not OptunaHub's DESampler. OptunaHub's samplers/differential_evolution is another implementation of DE/rand/1/bin, which this class does not follow, so a study moved over from it gets other defaults and other trials (registry pinned in refs/optunahub-registry):

OptunaHub `DESampler`	This class
`population_size="auto"`: `max(20, 10d)` below 5 dimensions, `max(30, 10d)` up to 30, then `max(50, 5d)`	`popsize`: `max(8, 4 + ⌊3 ln d⌋)`
`F=0.8`	`f = 0.8`
`CR=0.7`	`cr = 0.9`
DE/rand/1/bin only	`strategy`: `"rand1bin"` (default) or `"best1bin"`
`search_space`, `debug`, `seed`	`seed`; the search space is inferred; plus `nStartupTrials`, `independentSampler`, `warnIndependentSampling`

GPSampler

Sampler using Gaussian-process-based Bayesian optimization — port of optuna/samplers/_gp/sampler.py: GPSampler. It fits a Matérn-5/2 ARD Gaussian process to the past trials (kernel hyperparameters by marginal-likelihood maximization with the Gamma / hand-crafted priors) and suggests the next parameters by maximizing an acquisition function with the mixed continuous/discrete optimizer (OptunaSharp.Gp.OptimMixed).

GP stays native (Tier N): no TorchSharp — the fit rides the hand-derived Matérn-5/2 marginal-likelihood gradient and the managed L-BFGS-B (OptunaSharp.Internal.LBfgsB), and the acquisition optimization the managed acqf gradient. The path is tolerance-exact vs Optuna (ULP drift from scipy's OpenBLAS BLAS + the erf/ucrt class in LogEI), the documented GP boundary — not bit-for-bit like the MT19937 samplers.

Scope (PORTING_PLAN Phase 3.2 part 3e — complete): every path is ported. Sequential (no running trials): single-objective unconstrained (OptunaSharp.Gp.LogEI), constrained (OptunaSharp.Gp.LogCEI), multi-objective (OptunaSharp.Gp.LogEHVI), and multi-objective constrained (OptunaSharp.Gp.LogCEHVI). With running trials (asked but not yet told): the q-batch acqfs built on the OptunaSharp.Gp.ConditionalGPRegressor — OptunaSharp.Gp.QLogEI, OptunaSharp.Gp.QLogCEI, OptunaSharp.Gp.QLogEHVI, and (since Optuna 5.0.0, #6804) OptunaSharp.Gp.QLogCEHVI for the multi-objective constrained case, which earlier versions handled with a Kriging-Believer OptunaSharp.Gp.LogCEHVI.

GridSampler

Sampler that performs exhaustive search over the define-and-run user-specified grids. Port of optuna/samplers/_grid.py: GridSampler.

With GridSampler, the trials suggest all combinations of parameters in the given search space during the study.

The C# analogue of Optuna's GridValueType = str | float | int | bool | None is CategoricalChoice, so the grid values carry the same Python-equality semantics (and the same type preservation — int vs float — that the journal format depends on).

JanusAgeMoea2CmixCptSampler

AGE-MOEA-2-CPT with JANUS-style CMIX candidate generation — the conditional-parameter sibling of JanusAgeMoea2CmixSampler and the JANUS-augmented sibling of AgeMoea2CptSampler. AGE-MOEA-2 keeps full control of its adaptive-geometry survival, the ConditionalParams rules keep branch-defined parameters in the evolved space, and — past a warm-up — a small fraction of children are replaced by an inverse-curvature Gaussian sample drawn from the recent window (component_losses residuals or a diagonal-quadratic scalar fallback).

When CMIX contributes nothing. The CMIX engine needs an all-numeric relative search space (it cannot transform categoricals). On a categorical-dominant or conditional-categorical space the gate never fires and this sampler is exactly AgeMoea2CptSampler — which is the intended, useful behavior for that setting (the value is AGE-MOEA-2's survival + CPT, not the infill). Reserve the CMIX layer for continuous multi-objective problems; see JanusMooCmixSampler for the CMIX parameters and the PCG64 RNG divergence.

JanusAgeMoea2CmixSampler: AGE-MOEA-2 with JANUS-style CMIX candidate generation — the many-objective (5/8/10) sibling of JanusMooCmixSampler. AGE-MOEA-2 keeps full control of its adaptive-geometry survival; past a warm-up a small fraction of children are replaced by an inverse-curvature Gaussian sample (from component_losses or a diagonal-quadratic scalar fallback). See JanusMooCmixSampler for the CMIX parameters and the RNG divergence.

JanusDeSampler

JANUS-DE: the training-free JANUS Jacobian infill on a Differential Evolution host. JANUS estimates a local Jacobian from the recent evaluation trace and bounded-injects a damped Gauss–Newton exploitation elite into each DE generation, while the DifferentialEvolutionSampler-style DE host keeps full control of selection. Because DE has no Gaussian proposal covariance, this is the paper's JANUS-GN configuration — the exploitation injection only, no CMIX exploration candidates. It is the host-agnostic complement to the CMA-hosted JanusSampler.

Residual coordinates (optional but recommended). If the objective decomposes into components f(x) = mean_i r_i(x), expose the residual vector on each trial via trial.SetUserAttr("component_losses", new double[]{ r1, r2, … }); JANUS then estimates the local Jacobian directly from those residuals. Without it, JANUS falls back to a local linear/quadratic fit on the recent window of scalar objective values (the constructor's pseudoMode option).

Beyond-core, in-memory state. Like JanusSampler, JANUS-DE holds its optimizer (JANUS window + DE population) in memory for the study's duration; it does not resume its infill state across process boundaries. Categorical and single-value distributions are handled by the independent sampler.

JanusMooCmixSampler

Multi-objective JANUS: NSGA-II selection with JANUS-style CMIX candidate generation. Past a warm-up, a small fraction (the constructor's cmixProb option) of children are replaced by an inverse-curvature Gaussian sample drawn from a metric estimated over the recent window — either from component_losses residuals (the recommended mode) or a diagonal-quadratic scalar fallback — centered on a crowding-ranked point. NSGA-II keeps full control of Pareto selection, so this matches or beats the bare NSGA-II host without a significant regression (the paper's headline MOO property). Port of the authors' janus_optuna.JanusMooCmixSampler (arXiv:2608.22862).

Residual coordinates (optional but recommended). If each objective decomposes into components, expose the residual vector per trial via trial.SetComponentLosses(new double[]{ … }); JANUS then estimates the local metric directly from those residuals. Without them it uses a diagonal-quadratic fit on the window's scalarized values.

Hosts. This is the NSGA-II-hosted sampler the reference ships. The same CMIX engine is host-agnostic; SMS-EMOA- and AGE-MOEA-2-hosted variants (for low- and many-objective suites) reuse it — see JanusSmsEmoaCmixSampler and JanusAgeMoea2CmixSampler.

JanusSampler

A plug-and-play, training-free JANUS sampler. JANUS estimates a local Jacobian from the recent evaluation trace and uses the same Jacobian for both a damped Gauss–Newton exploitation candidate (bounded-injected into the population) and a trace-preserving inverse-curvature exploration metric (a fraction of the offspring), while a CMA-ES host keeps full control of selection, covariance adaptation, and step-size control. Port of the authors' reference janus_optuna.JanusSampler (arXiv:2608.22862; https://github.com/hongyuanyu/JANUS).

JANUS-GN vs full JANUS. Passing cmixCandidateFrac: 0 (or cmixMode: "none") disables the CMIX exploration candidates, leaving only the Gauss–Newton exploitation injection — the paper's conservative JANUS-GN variant. The default is full JANUS (GN + CMIX).

Residual coordinates (optional but recommended). If the objective decomposes into components f(x) = mean_i r_i(x), expose the residual vector on each trial via trial.SetUserAttr("component_losses", new double[]{ r1, r2, … }); JANUS then estimates the local Jacobian directly from those residuals, where its geometric signal is sharpest. Without it, JANUS falls back to a local linear/quadratic fit on the recent window of scalar objective values (the paper's BBOB "scalar residual coordinate" mode — see pseudoMode).

Beyond-core, in-memory state. Unlike CmaEsSampler, which serializes its optimizer into trial system attrs (for RDB / distributed resume), JANUS holds its optimizer in memory for the study's duration — matching the reference implementation. A JANUS study therefore does not resume its infill state across process boundaries. Categorical and single-value distributions are handled by the independent sampler, as in CmaEsSampler.

JanusSmsEmoaCmixSampler: SMS-EMOA with JANUS-style CMIX candidate generation — the hypervolume-hosted sibling of JanusMooCmixSampler, for low-objective (2–3) suites. SMS-EMOA keeps full control of its steady-state hypervolume selection; past a warm-up a small fraction of children are replaced by an inverse-curvature Gaussian sample (from component_losses or a diagonal-quadratic scalar fallback). See JanusMooCmixSampler for the CMIX parameters and the RNG divergence.

LazyRandomState

Lazy Random State class. Port of optuna/samplers/_lazy_random_state.py.

This is a class to initialize the random state just before use to prevent duplication of the same random state when a sampler instance is copied.

Every sampler consumes randomness through this wrapper over NumSharp's NumPy-exact MT19937 RandomState — never Random — which is what makes differential testing against Python bit-exact (plan §2 RNG discipline).

LmCmaSampler

A sampler driving the paper-identical OptunaSharp.Samplers.CmaEs.LmCma optimizer — Limited-Memory CMA-ES (Ilya Loshchilov, "LM-CMA: an Alternative to L-BFGS for Large Scale Black-box Optimization", Evolutionary Computation 2015; arXiv:1511.00221). LM-CMA stores only m ≪ n direction vectors and reconstructs the Cholesky factor on the fly (O(mn) memory and per-sample time, no eigendecomposition), which targets very high-dimensional search spaces where full OptunaSharp.Samplers.CmaEs.Cma's O(n²) covariance is impractical. On low-dimensional / ill-conditioned problems it converges more slowly than full CMA-ES — that is the documented large-scale trade-off, not a defect.

Distinct from CmaEsSampler. This is a separate, standalone sampler, not a flag on CmaEsSampler: LM-CMA shares none of that sampler's with_margin / use_separable_cma / warm-start / restart machinery, and reusing it would force-fit features the paper does not define. LM-CMA is not part of the Optuna ecosystem (no cmaes class, no Optuna sampler, no OptunaHub package), so — unlike the other CMA variants — this is a native paper-faithful implementation validated by convergence, with no upstream to bit-match.

Two paper concessions the per-trial protocol forces (both documented on the optimizer): (1) LM-CMA (the paper) is unconstrained; Optuna needs in-box params, so the sampler clamps each asked candidate to the transformed bounds and the next tell re-derives x from the stored (clamped) params — the optimizer learns from the actually-evaluated points. (2) Optuna asks one candidate per trial from freshly-restored state, which cannot carry the intra-generation mirror pairing (Alg. 7 l.9–10), so the sampler drives the non-mirrored generation via AskCandidate(int), passing the true generation position so SelectSubset's k==1 boost still fires once per generation.

Categorical parameters and single-value distributions are handled by the independent sampler, as in CmaEsSampler. The optimizer needs n > 1; a single-dimensional relative space falls back to independent sampling.

MoeaDSampler

Multi-objective sampler using the MOEA/D algorithm (Zhang & Li, 2007): a decomposition-based evolutionary algorithm that splits the multi-objective problem into a spread of scalar subproblems (one per Das-Dennis weight vector) and optimizes them cooperatively through neighborhood mating and decomposition-based replacement. The scalarizing function is Tchebycheff by default (MoeadScalarization also offers PBI and weighted sum).

Beyond-core. Optuna has no MOEA/D. OptunaHub's samplers/moead (MOEADSampler) is another implementation of the algorithm, which this class does not follow: it ports pymoo's MOEAD, so a study moved over from the hub package gets other defaults and other trials (map below). Like SmsEmoaSampler it is validated by convergence and achieved hypervolume, not bit-matched to an upstream. The crossover/mutation are the shared NSGA operators (SBXCrossover + PolynomialMutation by default), so only the selection and replacement differ from the NSGA-II family.

OptunaHub's MOEADSampler and this class (registry pinned in refs/optunahub-registry):

OptunaHub `MOEADSampler`	This class
`population_size=100`	`populationSize = 50`, rounded down to a Das-Dennis lattice size (the hub uses the number as given)
`n_neighbors=None` (then `population_size // 10`)	`nNeighbors = 20`
`scalar_aggregation_func="tchebycheff"` (also `"weighted_sum"`, `"PBI"`)	`scalarization`: Tchebycheff by default, plus PBI and weighted sum
`crossover=None`, meaning `UniformCrossover(swapping_prob)`	`crossover`: SBXCrossover by default
`mutation=None`, meaning `UniformMutation()`	`mutation`: PolynomialMutation by default
`crossover_prob=0.9` (otherwise a random parent is copied)	none: every offspring is crossed
`mutation_prob=None` (1 / number of parameters), `swapping_prob=0.5`, `seed`	`mutationProb`, `swappingProb`, `seed`, same defaults
(mates within the neighborhood)	`neighborSelectionProb = 0.9` (pymoo's `prob_neighbor_mating`) and `nReplacements = 2`

In-memory state. The N-solution population, its ideal point and the folded-offspring bookkeeping are reconstructed incrementally and held in memory for the study's process lifetime, so a MOEA/D study does not resume across process boundaries (same concession as SmsEmoaSampler). Categorical and single-value distributions are handled by the independent sampler.

Actual population size. The structured Das-Dennis lattice has a fixed size for each (objectives, partitions) pair, so the sampler uses the largest lattice whose size does not exceed the requested populationSize; the effective N (and hence the number of subproblems) can therefore be slightly below populationSize.

NSGAIIICptSampler

NSGA-III-CPT ("NSGA-III with Categorical Parameter Trees") — an OptunaSharp original that extends NSGA-III to optimize conditional (define-by-run, branch-defined) hyperparameters. It is a standalone copy of NSGAIIISampler: same many-objective genetic core (reference-point niching via NSGAIIIElitePopulationSelectionStrategy, the shared crossovers, mutations and ranking), plus the conditional-parameter machinery that NSGAIIISampler deliberately no longer carries.

Why it exists. Optuna's GA samplers evolve only the intersection search space, so a parameter suggested under one branch (if optimizer == "adam": beta1 = suggest(...)) drops out of the evolved space after the first trial that lacks it and is resampled at random ever after — it gets no crossover, mutation or inheritance. NSGA-III-CPT takes a declarative ConditionalParams rule set that says which parameters are conditional on which activator values; those parameters then stay in the relative search space (NsgaiiiCptSearchSpace), are crossed over, mutated and inherited whenever active (NsgaiiiCptChildGenerationStrategy / OptunaSharp.Samplers.NsgaiiiCpt.NsgaiiiCptCrossover), and are simply absent whenever not. This is the tree of categorically-gated parameters the "CPT" name denotes. See the NSGA-III-CPT documentation page (docs/website-src/docs/our-methods/nsga-iii-cpt.md).

Relationship to NSGA-III. Constructed with conditionalParams: null this sampler is behaviorally identical to NSGAIIISampler (the CPT search space and child generation both fall back to the shared Optuna-parity fold), so it is a strict superset. The conditional behavior is a deliberate divergence from Optuna and has no upstream analog; the underlying NSGA-III algorithm is the bit-exact port documented on NSGAIIISampler.

NSGAIIISampler

Multi-objective sampler using the NSGA-III algorithm. Port of optuna/samplers/_nsgaiii/_sampler.py: NSGAIIISampler (experimental since Optuna 3.2.0).

NSGA-III ("Nondominated Sorting Genetic Algorithm III") is a modification of NSGAIISampler for many-objective optimization. It reuses the whole NSGA-II genetic machinery (BaseGASampler, child generation, crossovers, mutations) and only swaps the elite-selection strategy: instead of crowding distance, the borderline front is truncated by reference-point niching (see NSGAIIIElitePopulationSelectionStrategy). See Deb & Jain (2014).

Parity. The default path stays in exact MT19937 lock-step with Optuna (generation attribution, parent cache, tournament/crossover/mutation draws, and the niching rng.shuffle), and objective-value niching is bit-exact: the hyperplane-fit matrix_rank/solve route through NumSharp's OpenBLAS LAPACK backend (byte-identical to NumPy), so the earlier ~1-ULP managed-LU drift is gone. Confirmed byte-for-byte over a 400-config differential plus the live parity seeds — see NSGAIIIElitePopulationSelectionStrategy.

Conditional parameters. To optimize conditional (define-by-run, branch-defined) hyperparameters — which otherwise fall out of the intersection search space and are resampled at random on every trial — use NSGAIIICptSampler (NSGA-III-CPT), the OptunaSharp variant that carries a ConditionalParams rule set. This sampler itself is a bit-exact Optuna port with no such extension.

NSGAIISampler

Multi-objective sampler using the NSGA-II algorithm. Port of optuna/samplers/nsgaii/_sampler.py: NSGAIISampler.

NSGA-II ("Nondominated Sorting Genetic Algorithm II") is a fast, elitist multi-objective genetic algorithm. See Deb et al. (2002).

Parity. The default (UniformCrossover + no mutation object) path is bit-exact against Optuna: uniform crossover copies parent values verbatim and mutated-away parameters are resampled by the independent RandomSampler (both MT19937). Every RNG draw runs through LazyRandomState, and the generation / parent-cache system attributes are byte-identical, so full study state matches. The arithmetic crossovers (BLX-α/SBX/vSBX/SPX) stay bit-exact; UNDXCrossover is tolerance-bounded for ≥2 numerical parameters (managed QR vs LAPACK — see its note) but keeps RNG lock-step.

OpenAiEsSampler

A sampler driving OpenAI Evolution Strategies (Salimans et al. 2017): an isotropic Gaussian N(mean, σ²I) whose mean ascends the natural gradient estimated from a population of mirrored, rank-shaped perturbations, stepped with Adam. It is a robust, embarrassingly-parallel, gradient-free optimizer that complements the covariance model of CmaEsSampler and the diagonal NES of SnesSampler.

Convergence character. OpenAI-ES uses a single global σ. With the defaults σ is annealed only slowly (the estool recipe), so within a typical Optuna budget the search settles to a neighborhood of the optimum rather than to machine precision — the natural regime for noisy, high-dimensional objectives. When you need fine convergence on a smooth objective, prefer SnesSampler (per-coordinate adaptive σ) or CmaEsSampler.

Unsupported parameters. Categorical parameters and single-valued distributions are sampled by the independent sampler, as in CmaEsSampler. Dynamic search spaces fall back to independent sampling with a warning.

PartialFixedSampler

Sampler with partially fixed parameters. Port of optuna/samplers/_partial_fixed.py: PartialFixedSampler (experimental since 2.4.0).

After several steps of optimization you can fix the value of some parameters and re-optimize the rest, delegating the unfixed parameters to a wrapped base sampler:

var study = Optuna.CreateStudy();
study.Optimize(Objective, nTrials: 10);

var fixedParams = new Dictionary<string, object?> { ["y"] = study.BestParams["y"] };
study.Sampler = new PartialFixedSampler(fixedParams, study.Sampler);
study.Optimize(Objective, nTrials: 10);

PsoSampler

A sampler driving a textbook-faithful global-best OptunaSharp.Samplers.ParticleSwarm.Pso Particle Swarm optimizer (Kennedy & Eberhart 1995; Shi & Eberhart 1998 inertia weight) through the Optuna protocol. PSO maintains a swarm of particles, each carrying a position, a velocity and a personal-best memory, and the swarm shares one global best; each particle's velocity is nudged toward its personal best and the global best (with an inertia term), giving a robust derivative-free scalar optimizer that complements CmaEsSampler and DifferentialEvolutionSampler as a benchmark baseline.

Bootstrapping. PSO needs an initial swarm before it can move particles, so the first max(nStartupTrials, populationSize) completed trials are drawn by the independent sampler and become the initial swarm (velocities start at rest); swarm moves are proposed from then on.

Two protocol concessions (as DifferentialEvolutionSampler): (1) the canonical PSO velocity/position update is unconstrained, so each asked candidate is clamped to the transformed unit box and the next generation recovers each particle's realized velocity as x_evaluated − x_previous (a velocity reset on bound contact — a standard PSO bound-handling rule). (2) The per-trial protocol asks one candidate per trial from freshly-restored state; each trial is stamped with its generation and its target index so the swarm's update pairs each evaluated position with the right particle once the generation completes.

Inertia schedule. Optuna is trial-budget-driven and does not expose a maximum generation to the sampler, so the linearly-decreasing inertia weight decays from w to wMin over inertiaDecayGenerations generations (constant at wMin thereafter). Categorical parameters and single-value distributions are handled by the independent sampler, as in CmaEsSampler.

Not OptunaHub's PSOSampler. OptunaHub's samplers/pso is another PSO, with a constant inertia, which this class does not follow, so a study moved over from it gets other defaults and other trials (registry pinned in refs/optunahub-registry):

OptunaHub `PSOSampler`	This class
`n_particles=10`	`popsize`: `max(10, 4 + ⌊3 ln d⌋)`
`inertia=0.5`, constant	`w = 0.9` decreasing linearly to `wMin = 0.4` over `inertiaDecayGenerations = 100`
`cognitive=1.5`, `social=1.5`	`c1 = c2 = 1.49445`
velocity clamped to the full range of each parameter	`maxVelocityRate = 0.2` of the unit box
`search_space`, `seed`	`seed`; the search space is inferred; plus `nStartupTrials`, `independentSampler`, `warnIndependentSampling`

QMCSampler

A Quasi Monte Carlo sampler that generates low-discrepancy sequences. Port of optuna/samplers/_qmc.py: QMCSampler (experimental since Optuna 3.0.0).

Optuna's sampler wraps scipy.stats.qmc. OptunaSharp reproduces the non-scrambled Sobol' and Halton sequences bit-for-bit through OptunaSharp.Internal.QuasiMonteCarlo. The search space is fixed by the first trial of the study; the first trial (and any parameter outside the relative search space) is drawn by the independent_sampler (a RandomSampler by default, so those draws stay on MT19937).

Scrambling (PORTING_PLAN Phase 2.12 — complete). Both randomizations are ported bit-exact vs scipy, seeded by numpy.random.default_rng(seed): scrambled Sobol' through OptunaSharp.Internal.ScrambledSobol (the linear-matrix-scramble + digital shift) and scrambled Halton through OptunaSharp.Internal.ScrambledHalton (Owen's digit permutations — scipy's _van_der_corput_permutations over NumSharp's Generator.shuffle). Every (qmc_type, scramble) combination is supported.

RandomEmbeddingSampler

A sampler implementing REMBO — Random EMbedding Bayesian Optimization (Wang et al. 2016). It draws a fixed random Gaussian matrix A ∈ ℝ^{D×d}, runs the paper's inner Bayesian optimizer (by default a squared-exponential GP with expected improvement maximized by DIRECT + CMA-ES and the Algorithm-3 adaptive length scale) in the small box Y = [−√d, √d]^d, and maps each latent point y to the study's real parameters through x = p_X(A·y) (a box-constrained projection). This lets BO scale to a search space of (in principle) enormous dimension D as long as the objective's effective dimensionality d_e is low — the invariance the paper proves and demonstrates up to a billion dimensions.

The one knob that matters: nEmbeddingDims (d). It must be at least the objective's (guessed) effective dimensionality for the embedding to contain the optimum; larger d raises the chance the box captures the optimum but slows the inner optimizer (Section 3.1). There is no universal default — it reflects a belief about the problem — so it is required.

Interleaved runs (nEmbeddings, k). A single embedding's box misses the optimum with some probability δ; running k independent embeddings drops that to δ^k (Section 3.1). With k > 1 trials are assigned to embeddings round-robin, each embedding keeping its own latent history.

How it fits the Optuna protocol. Every real parameter is produced through SampleIndependent(Study, FrozenTrial, string, BaseDistribution) from a single latent point sampled once per trial — so all real parameters of a trial come from one coherent y, and even the very first trial is embedded (its observation feeds the inner optimizer). By default the latent point is chosen by REMBO's paper-faithful optimizer — a squared-exponential GP with expected improvement maximized by DIRECT + CMA-ES and the Algorithm-3 adaptive length scale — fit on the finished trials' latent points; an overridden inner sampler is driven over a private in-memory shadow study of them instead. Either way the latent point is persisted in the real trial's system attributes so the run can be rebuilt after resumption. Float and integer parameters are embedded (integers/steps/log honored via the standard de-normalization); categorical and single-valued parameters are delegated to an independent sampler, as in CmaEsSampler / TurboSampler. Single-objective only.

Beyond-core, convergence-validated. REMBO has no Optuna counterpart and its inner GP is stochastic, so — like TurboSampler and LmCmaSampler — it is validated by convergence (embedding recovery on a low-effective-dimension objective) and same-seed determinism under sequential optimization, not by byte-parity.

RandomSampler

Sampler using random sampling. Port of optuna/samplers/_random.py: RandomSampler.

This sampler is based on independent sampling. See also BaseSampler for more details of 'independent sampling'.

RveaSampler

Multi-objective sampler using the RVEA algorithm (Cheng et al., 2016): a reference-vector-guided evolutionary algorithm for many-objective optimization. It reuses the whole NSGA-II genetic machinery (BaseGASampler, child generation, crossovers, mutations) and swaps the elite-selection strategy for angle-penalized-distance (APD) survival (OptunaSharp.Samplers.Rvea.RveaApdSurvival): each individual is associated with its nearest reference vector, and within every reference vector the survivor is the one whose distance to the ideal point is least after a generation-growing angle penalty — an explicit diversity→convergence shift that suits three-and-more-objective fronts. The reference vectors are periodically re-scaled to the current objective ranges.

Beyond-core. RVEA has no Optuna / OptunaHub counterpart; like SmsEmoaSampler it is validated by convergence and achieved hypervolume, not bit-matched to an upstream. The default operators are RVEA's classic SBXCrossover (η=30) + PolynomialMutation (η=20); mating uses the shared NSGA-II child generation, so only the survival differs from the NSGA-II family.

Generation horizon. The APD penalty grows with (generation / nMaxGenerations)^α and the reference vectors adapt every ceil(nMaxGenerations · adaptFreq) generations. Optuna is trial-budget-driven and does not expose a maximum generation to the sampler, so nMaxGenerations sets that schedule's horizon explicitly (the ratio is clamped to 1 beyond it).

Reference-vector count. The structured Das-Dennis lattice has a fixed size for each (objectives, partitions) pair, so — unless explicit reference points are supplied — the effective population size is set to the largest lattice size not exceeding the requested populationSize.

SaasboSampler

A sampler using SAASBO — high-dimensional Bayesian optimization with sparse axis-aligned subspaces (Eriksson & Jankowiak 2021). Each trial it fits an RBF ARD Gaussian process under the SAAS prior — a half-Cauchy sparsity prior on the per-dimension inverse-squared length scales that "turns off" most dimensions unless the data demands otherwise (automatic relevance determination) — and proposes the next point by maximizing the expected improvement averaged over posterior hyperparameter samples. This is the paper's answer to the curse of dimensionality: work directly in the full space, but let a sparsity-inducing function prior keep the surrogate parsimonious enough to infer from a few hundred queries.

Inference. The default is the No-U-Turn Sampler (Nuts, the paper's "gold standard"), which averages EI over a bag of L posterior draws; Map is a faster point-estimate alternative (Adam over a small shrinkage grid + leave-one-out selection). The single prior hyperparameter is α (global shrinkage, default 0.1); every other knob is purely computational.

Cost footgun. SAASBO is deliberately expensive: the paper's default NUTS budget (512 warmup + 256 samples, thinned to 16) runs Hamiltonian Monte Carlo over an O(N³D) gradient per leapfrog step, and the default EI maximization screens Q = 5000 Sobol candidates before three L-BFGS-B polishes — appropriate when the objective is expensive to evaluate (the paper targets a few hundred queries) but heavy otherwise. Lower numWarmup/numSamples/numCandidates (or use Map) to trade fidelity for speed. Best for N ≲ 500 observations.

Continuous / integer only. The SAAS GP models a continuous [0, 1]^D box, so the relative search space is Float/Int only; categorical and single-valued parameters are delegated to the independent sampler, exactly as in TurboSampler / CmaEsSampler. Dynamic search spaces fall back to independent sampling with a warning. Single-objective only.

State is reconstructed, not persisted. Like TurboSampler, SAASBO stores no optimizer blob: each trial re-fits the GP to the completed observations, so a study resumes from storage with no serialized state. The one thing it writes is the Sobol row each initialization trial took (the system attr "saasbo:init"), so trials asked at the same time (Optimize(nJobs)) take distinct rows instead of all receiving the row the completed count names.

Conformance to the paper (audited; see SaasboPaperClaimsTests). The RBF kernel (Eqn 1), closed-form posterior (Eqns 4-5), SAAS prior (Eqn 8) with the reparameterization (Eqn 12), the EI acquisition and its sample-averaging (Eqns 6-7, 10), the NUTS budget (§A.1), the MAP recipe (§A.2), and the EI-maximization strategy (§B) are all implemented as stated; the empirical claims of §5.2 (relevant-subspace identification, α controlling shrinkage) and §4.1/§5.7 (adaptive sparsity — more data turns on more dimensions) are asserted on seeded chains. Four deliberate, documented scope choices differ from the paper's full generality, none of them a correctness gap: (1) the RBF kernel only — the paper's default — not the optional Matérn-5/2; (2) a fixed tiny noise (the paper's noise-free primary setting), so the noisy-objective extension (a LN(0,10²) prior on σ²) is not modeled; (3) standardized targets (a numerical convention shared with the reference implementation and the rest of the GP stack); and (4) SAASBO-MAP maximizes the density in the constrained parameter (the paper's literal Eqn 13, "maximize U_s w.r.t. ψ"), not the unconstrained-space mode. The MAP shrinkage grid follows the authoritative §A.2 (S=4, {1,1e-1,1e-2,1e-3}), resolving the main text's smaller §4.2.2 list. Parity is behavioral (HMC is stochastic), not byte-exact vs NumPyro/BoTorch — the TuRBO/REMBO validation tier.

SmsEmoaSampler

Multi-objective sampler using the SMS-EMOA algorithm (Beume, Naujoks & Emmerich, 2007): a steady-state (μ+1) evolutionary algorithm whose survival operator removes the individual with the least hypervolume contribution in its worst nondomination front. Where NSGA-II breaks ties within a front by crowding distance, SMS-EMOA breaks them by the S-metric (dominated hypervolume) — so it steers directly toward a high-hypervolume Pareto approximation.

Beyond-core. SMS-EMOA has no Optuna / OptunaHub counterpart; like LmCmaSampler it is validated by convergence and achieved hypervolume, not bit-matched to an upstream. The crossover/mutation are the shared NSGA-II operators (SBXCrossover + PolynomialMutation by default), so behavior on the variation side matches the NSGA-II family; only the survival rule differs.

In-memory state. The μ-survivor population is reconstructed incrementally and held in memory for the study's duration, so a SMS-EMOA study does not resume its population across process boundaries (same concession as JanusSampler). Categorical and single-value distributions are handled by the independent sampler.

SnesSampler

A sampler driving Separable Natural Evolution Strategies (Wierstra et al. 2014): a diagonal Gaussian with a per-coordinate mean and scale, adapted by natural-gradient steps. Because σ is adapted per coordinate, SNES converges to high precision without the fixed-σ floor of OpenAiEsSampler, while costing only O(d) per sample — the practical NES for higher-dimensional continuous hyperparameter spaces where full-covariance CmaEsSampler becomes expensive. The price is an assumption of (approximate) coordinate independence.

TPESampler

Sampler using the TPE (Tree-structured Parzen Estimator) algorithm — Optuna's default sampler. Port of optuna/samplers/_tpe/sampler.py: TPESampler.

On each trial, for each parameter, TPE fits one Gaussian Mixture Model l(x) to the parameter values of the best trials, and another GMM g(x) to the rest, then picks the value maximizing l(x)/g(x).

Scope (PORTING_PLAN Phase 2.8/2.9). The univariate (independent), the non-group multivariate, and (Phase 2.9) the group-decomposed multivariate (group=true) paths are implemented — so a default TPESampler works for single- and multi-objective studies. Multi-objective splitting inherits the Phase 2.10 HSSP tie-break gap (an explicit target avoids it).

[Misaligned] erf/ucrt tolerance. The sampled values and acquisition densities route through OptunaSharp.Internal.TruncNorm → OptunaSharp.Internal.Erf (FreeBSD s_erf), ~1-3 ULP from Python's math.erf (ucrt). So TPE agrees with Optuna to a few ULP, not bit-for-bit — the same bound documented for PED-ANOVA. RNG draw order and count are identical, so the study trajectory stays aligned. See also the argsort tie-break note on OptunaSharp.Samplers.Tpe.ParzenEstimator.

TurboSampler

A sampler using TuRBO — trust-region Bayesian optimization (Eriksson et al. 2019). It fits a local Matérn-5/2 GP to the observations of the current local run, keeps a trust region (a length-scale-weighted hyperrectangle) around the incumbent, and proposes the next point by Thompson sampling over a Sobol perturbation set inside that region. The region expands after consecutive improvements, shrinks after stalls, and restarts once it collapses — the mechanism that lets TuRBO keep making progress in higher dimension where a single global GP (see GPSampler) over-explores.

TuRBO-1 vs TuRBO-m. With the default numTrustRegions = 1 this is TuRBO-1, a single trust region restarted sequentially. With numTrustRegions = m > 1 it is TuRBO-m: m trust regions run simultaneously, each an independent TuRBO-1 with its own local GP, incumbent, and resize counters, and an implicit multi-armed bandit (a Thompson sample drawn from every region, global best wins) allocates each evaluation to the most promising region. TuRBO-m concentrates the budget on the best of several basins (§3.7) but divides that budget across the regions, so it is not uniformly better — it wins on multimodal/deceptive problems and can tie or slightly lose on easy ones; the paper accordingly reports TuRBO-1 and TuRBO-5/-20 by regime.

Beyond-core, convergence-validated. Optuna itself has no TuRBO. OptunaHub has one, by the Optuna team (samplers/turbo, TuRBOSampler, since 2025-09), which this class does not follow: it ports the paper (Eriksson et al. 2019), so a study moved over from the hub package gets other defaults and other trials (map below; QA-70, owner's decision of 2026-11-12 to keep this port). So — like LmCmaSampler and DifferentialEvolutionSampler — it is validated by convergence rather than parity. GP fitting + Thompson sampling ride the same native GP stack as GPSampler (hand-derived Matérn-5/2 marginal-likelihood gradient, managed L-BFGS-B, OpenBLAS Cholesky), so results are tolerance-reproducible on a platform, not bit-exact vs BoTorch.

OptunaHub's TuRBOSampler and this class (registry pinned in refs/optunahub-registry):

OptunaHub `TuRBOSampler`	This class
`n_trust_region=5`	`numTrustRegions = 1` (TuRBO-1; pass 5 for TuRBO-5)
`n_startup_trials=4`	`nInit`: `2·d` Sobol points per (re)start
`failure_tolerance=5`	`failureTolerance`: the paper's `d`
`success_tolerance=3`, `init_length=0.8`, `max_length=1.6`, `min_length=0.5**7`	`successTolerance = 3`, `lengthInit = 0.8`, `lengthMax = 1.6`, `lengthMin = 1/128`: the same values
`seed`, `independent_sampler`, `deterministic_objective`, `warn_independent_sampling`	the same names in C# casing; plus `numCandidates`, `successMargin`, `minimumNoise`

Continuous / integer only. The trust-region box is a continuous construct, so the relative search space is Float/Int only; categorical and single-valued parameters are delegated to the independent sampler, exactly as in CmaEsSampler. Dynamic search spaces fall back to independent sampling with a warning. Single-objective only.

State is reconstructed, not persisted (with one small exception for TuRBO-m). TuRBO-1 stores no optimizer blob: the trust-region length, resize counters, and restart boundaries are recomputed each trial by a deterministic fold over the completed-trial objective sequence (ReconstructState(double[], int)), so a study resumes from storage with no serialized state. TuRBO-m adds the minimum needed to fold each region: because a trial's region is the seeded competition winner (a function of which prior trials landed in which region, not of the values alone), m > 1 persists one small per-trial region label (system attr "turbo:tr"); each region then folds exactly as TuRBO-1 over its labeled subsequence.

VkdCmaSampler

A sampler driving the OptunaSharp.Samplers.CmaEs.VkdCma optimizer — VkD-CMA-ES (Akimoto & Hansen, GECCO/PPSN 2016), a port of Akimoto's own GaussVkDSampler. VkD-CMA restricts the covariance to a rank-k term plus a diagonal and adapts the diagonal, the low-rank basis, AND the rank k online — cost O(N k² + k³) per generation, the middle point between a diagonal-only model (CmaEsSampler with useSeparableCma) and full CmaEsSampler's O(N²) covariance. It targets high-dimensional problems whose covariance is well-approximated by a few dominant directions, and learns how many directions (k) that is as it runs.

Distinct from CmaEsSampler and LmCmaSampler. Like LmCmaSampler, this is a separate standalone sampler, not a flag on CmaEsSampler, and it shares none of that sampler's with_margin / use_separable_cma / warm-start / restart machinery. VkD-CMA is not part of the Optuna ecosystem (no cmaes class, no Optuna sampler, no OptunaHub package), so — like LmCmaSampler — it is a native, reference-faithful implementation validated by convergence, with no upstream to bit-match. It sits beside LmCmaSampler in the restricted-covariance family (both approximate the covariance; LM-CMA with stored direction vectors, VkD-CMA with an adapting rank-k+diagonal factorization).

Two concessions the per-trial protocol forces (both documented on the optimizer): (1) The optimizer is unconstrained; Optuna needs in-box params, so the sampler clamps each asked candidate to the transformed bounds and the next tell re-derives x from the stored (clamped) params — the optimizer learns from the actually-evaluated points. (2) VkD-CMA adapts its step-size by TPA, which injects a mirrored pair as a generation's first two samples and recovers them by direction; Optuna asks one candidate per trial from freshly-restored state, so the sampler drives the generation through AskCandidate(int), passing the true generation position — positions 1 and 2 emit the exact ±dx directions (with independent lengths, which TPA's direction recovery does not depend on).

XnesSampler

A sampler driving Exponential Natural Evolution Strategies (Wierstra et al. 2014): a full-covariance Gaussian factored as A = σ·B and adapted through the matrix exponential of a natural-gradient direction — the NES most closely related to CMA-ES's rank-µ update. It captures correlations between parameters that the diagonal SnesSampler cannot, at O(d²)–O(d³) cost per generation.

When to prefer alternatives. OptunaSharp already ships mature full-covariance adaptation (CmaEsSampler, VkdCmaSampler, LmCmaSampler); xNES is provided for completeness of the NES family. For most correlated continuous problems CmaEsSampler is the better-tuned choice, and SnesSampler is cheaper in high dimension.

Enums

RandomEmbeddingKernel: Which of REMBO's two kernels (Wang et al. 2016, §3.2) the inner Bayesian optimizer uses.

SaasboInferenceMethod: The posterior-inference strategy SAASBO uses to fit the SAAS GP each trial (Eriksson & Jankowiak 2021, §4.2). The choice trades inference fidelity for runtime and does not change the SAAS function prior itself.

Table of Contents

Namespace OptunaSharp.Samplers

Classes

Enums