Class WilcoxonPruner

Namespace

OptunaSharp.Pruners

Assembly

OptunaSharp.dll

Pruner based on the Wilcoxon signed-rank test. Port of optuna/pruners/_wilcoxon.py: WilcoxonPruner (experimental since 3.6.0).

This pruner performs the Wilcoxon signed-rank test between the current trial and the current best trial, and stops whenever the pruner is sure up to a given p-value that the current trial is worse than the best one.

This pruner is effective for optimizing the mean/median of some (costly-to-evaluate) performance scores over a set of problem instances. When you use this pruner, you must call Trial.Report(value, step) for each step (instance id) with the evaluated value; the instance id may not be in ascending order. Unlike other pruners, the reported values need not converge to the real value.

This pruner cannot handle infinity or NaN values; trials containing those values are never pruned. The scipy dependency is replaced by a self-contained signed-rank test (plan 2.3), verified against scipy.

public class WilcoxonPruner : BasePruner

Inheritance

object

BasePruner

WilcoxonPruner

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Constructors

WilcoxonPruner(double, int)

Configures a paired signed-rank test against the current best trial.

public WilcoxonPruner(double pThreshold = 0.1, int nStartupSteps = 2)

Parameters

pThreshold double

The p-value threshold for pruning, between 0 and 1. A trial will be pruned whenever the pruner is sure up to the given p-value that the current trial is worse than the best trial. The larger this value is, the more aggressive pruning will be performed. Note that this pruner repeatedly performs statistical tests with increasing samples; to get the nominal false-positive rate, specify the Pocock-corrected p-value.

nStartupSteps int

The number of steps before which no trials are pruned. Pruning starts only after you have nStartupSteps steps of available observations for comparison between the current trial and the best trial. Note the trial is not pruned at the first and second steps even if set to 0 or 1, due to the lack of enough data.

Exceptions

ValueError

The startup count is negative or the p-value threshold is outside [0, 1].

Methods

Prune(Study, FrozenTrial)

Judge whether the trial should be pruned based on the reported values.

Note that this method is not supposed to be called by library users. Instead, Report(double, long) and ShouldPrune() provide user interfaces to implement pruning mechanism in an objective function.

public override bool Prune(Study study, FrozenTrial trial)

Parameters

study Study

Study object of the target study.

trial FrozenTrial

FrozenTrial object of the target trial. Take a copy before modifying this object.

Returns

bool

A boolean value representing whether the trial should be pruned.

Exceptions

RuntimeError

The study has multiple objectives and cannot supply a scalar best trial or direction.

See Also

Official Optuna WilcoxonPruner reference.