Class TPESampler

Namespace

OptunaSharp.Samplers

Assembly

OptunaSharp.dll

Sampler using the TPE (Tree-structured Parzen Estimator) algorithm — Optuna's default sampler. Port of optuna/samplers/_tpe/sampler.py: TPESampler.

On each trial, for each parameter, TPE fits one Gaussian Mixture Model l(x) to the parameter values of the best trials, and another GMM g(x) to the rest, then picks the value maximizing l(x)/g(x).

Scope (PORTING_PLAN Phase 2.8/2.9). The univariate (independent), the non-group multivariate, and (Phase 2.9) the group-decomposed multivariate (group=true) paths are implemented — so a default TPESampler works for single- and multi-objective studies. Multi-objective splitting inherits the Phase 2.10 HSSP tie-break gap (an explicit target avoids it).

[Misaligned] erf/ucrt tolerance. The sampled values and acquisition densities route through OptunaSharp.Internal.TruncNorm → OptunaSharp.Internal.Erf (FreeBSD s_erf), ~1-3 ULP from Python's math.erf (ucrt). So TPE agrees with Optuna to a few ULP, not bit-for-bit — the same bound documented for PED-ANOVA. RNG draw order and count are identical, so the study trajectory stays aligned. See also the argsort tie-break note on OptunaSharp.Samplers.Tpe.ParzenEstimator.

public class TPESampler : BaseSampler

Inheritance

object

BaseSampler

TPESampler

Inherited Members

BaseSampler.ToString()

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

Constructors

TPESampler(bool?, double?, bool?, bool?, int, int, Func<int, int>?, Func<int, double[]>?, int?, bool?, bool, bool?, bool, Func<FrozenTrial, IEnumerable<double>>?)

Configures density estimation, startup sampling, and optional joint search-space groups.

public TPESampler(bool? considerPrior = null, double? priorWeight = null, bool? considerMagicClip = null, bool? considerEndpoints = null, int nStartupTrials = 10, int nEiCandidates = 24, Func<int, int>? gamma = null, Func<int, double[]>? weights = null, int? seed = null, bool? multivariate = null, bool group = false, bool? warnIndependentSampling = null, bool constantLiar = true, Func<FrozenTrial, IEnumerable<double>>? constraintsFunc = null)

Parameters

considerPrior bool?

Deprecated (v4.3.0). Falls back to true.

priorWeight double?

Deprecated (v4.9.0). Falls back to 1.0.

considerMagicClip bool?

Deprecated (v4.9.0). Falls back to true.

considerEndpoints bool?

Deprecated (v4.9.0). Falls back to false.

nStartupTrials int

Random sampling is used until this many trials finish.

nEiCandidates int

Number of candidates for the expected-improvement computation.

gamma Func<int, int>

Deprecated (v4.9.0). Splits finished trials into below/above.

weights Func<int, double[]>

Deprecated (v4.9.0). Weight of each finished trial.

seed int?

Seed for the random number generator.

multivariate bool?

If true, multivariate TPE is used. If null (default), it is true for single-objective and false for multi-objective optimization.

group bool

If this and multivariate are true, the multivariate TPE with a group-decomposed search space is used: the search space is partitioned into maximal subspaces from the past trials and each subspace is sampled jointly by multivariate TPE. Requires multivariate to not be false (experimental since v2.8.0).

warnIndependentSampling bool?

Deprecated (v4.9.0). Falls back to false.

constantLiar bool

Penalize running trials to avoid nearby suggestions. Defaults to true.

constraintsFunc Func<FrozenTrial, IEnumerable<double>>

Deprecated (v5.0.0). Optional constraint function evaluated after each trial.

Exceptions

ValueError

Grouping is enabled while multivariate estimation is explicitly disabled.

Methods

AfterTrial(Study, FrozenTrial, TrialState, IReadOnlyList<double>?)

Trial post-processing: called after the objective function returns and right before the trial is finished and its state is stored.

public override void AfterTrial(Study study, FrozenTrial trial, TrialState state, IReadOnlyList<double>? values)

Parameters

study Study

Study that owns the trial being finished.

trial FrozenTrial

Trial snapshot before its terminal state is written to storage.

state TrialState

Terminal state being recorded: complete, pruned, or failed.

values IReadOnlyList<double>

Resulting trial values in objective order; non-null for a successful trial.

Exceptions

Exception

A configured after-trial or constraint callback fails; its exception propagates.

BeforeTrial(Study, FrozenTrial)

Trial pre-processing: called during trial initialization, just before the InferRelativeSearchSpace(Study, FrozenTrial) call.

public override void BeforeTrial(Study study, FrozenTrial trial)

Parameters

study Study

Study that owns the newly initialized trial.

trial FrozenTrial

New trial snapshot available before search-space inference.

InferRelativeSearchSpace(Study, FrozenTrial)

Infer the search space that will be used by relative sampling in the target trial.

This method is called right before SampleRelative(Study, FrozenTrial, OrderedDict<string, BaseDistribution>), and the search space returned by this method is passed to it. The parameters not contained in the search space will be sampled by using SampleIndependent(Study, FrozenTrial, string, BaseDistribution).

public override OrderedDict<string, BaseDistribution> InferRelativeSearchSpace(Study study, FrozenTrial trial)

Parameters

study Study

Study supplying trial history, directions, and storage.

trial FrozenTrial

Current trial snapshot; copy it before modifying its contents.

Returns

OrderedDict<string, BaseDistribution>

Parameter distributions eligible for joint sampling, in sampler-defined order.

ReseedRng()

Reseed sampler's random number generator.

Parallel optimization calls this hook before a worker's sequential trial loop. Concrete samplers refresh their random streams so workers do not deliberately reuse a seeded sequence. OptunaSharp workers share the study's sampler instance; this hook does not clone it or make concurrent sampling reproducible.

public override void ReseedRng()

SampleIndependent(Study, FrozenTrial, string, BaseDistribution)

Sample a parameter for a given distribution.

This method is called only for the parameters not contained in the search space returned by SampleRelative(Study, FrozenTrial, OrderedDict<string, BaseDistribution>). This method is suitable for sampling algorithms that do not use relationship between parameters such as random sampling and TPE.

public override object? SampleIndependent(Study study, FrozenTrial trial, string paramName, BaseDistribution paramDistribution)

Parameters

study Study

Study supplying trial history, directions, and storage.

trial FrozenTrial

Current trial snapshot; copy it before modifying its contents.

paramName string

Name of the parameter being suggested.

paramDistribution BaseDistribution

Distribution defining the parameter's domain and scale.

Returns

object

A value in the distribution's external representation, including null categorical choices.

SampleRelative(Study, FrozenTrial, OrderedDict<string, BaseDistribution>)

Sample parameters in a given search space.

The trial invokes this method lazily when it first needs relative parameters and caches the result for its remaining suggestions. This method is suitable for sampling algorithms that use relationship between parameters such as Gaussian Process and CMA-ES.

public override OrderedDict<string, object?> SampleRelative(Study study, FrozenTrial trial, OrderedDict<string, BaseDistribution> searchSpace)

Parameters

study Study

Study supplying trial history, directions, and storage.

trial FrozenTrial

Current trial snapshot; copy it before modifying its contents.

searchSpace OrderedDict<string, BaseDistribution>

Distributions returned by InferRelativeSearchSpace(Study, FrozenTrial).

Returns

OrderedDict<string, object>

Jointly sampled external parameter values; omitted parameters use independent sampling.

See Also

Official Optuna TPESampler reference.