Methods

This section is the per-method deep-dive reference for OptunaSharp: one page per algorithm, each with its paper citation, the idea (with pseudocode), the OptunaSharp class and a runnable example, its lineage, and cross-links. It complements the Guides (which explain how to use a family) by explaining how each method works.

Why "Methods" and not "Algorithms"

Everything here is a named, published algorithm — but the set spans two tiers, which is why the umbrella is "methods" rather than the flatter "algorithms":

Tier 1 — methods you choose. The things you plug into a Study (samplers, pruners) or run to analyze one (importance evaluators, terminators).
Tier 2 — building blocks. The operators (crossovers, mutations) and numerical kernels (hypervolume, reference directions, QMC sequences, GP acquisition functions, CMA optimizers) that Tier 1 is composed of.

How this list is ordered

Within each role, methods are grouped by lineage — papers that build on one another sit together — and each group is sorted newest first. So a reader sees each family as a chain from its latest refinement back to its origin.

✅ — full deep-dive page.
🚧 — stub (header + paper link); the write-up is being filled in, in batches.

Every method below already ships in code today (see the API reference); 🚧 marks only the documentation, not the feature.

Tier 1 — methods you choose

Samplers — search methods

JANUS infill family — a single online-Jacobian-aligned infill idea specialized per host optimizer.

	Method	Paper
🚧	JANUS	Yu et al., arXiv:2608.22862 (2026)
🚧	JANUS-DE	Yu et al., arXiv:2608.22862 (2026)
🚧	JANUS-MOO-CMIX	Yu et al., arXiv:2608.22862 (2026)
🚧	JANUS-SMS-EMOA-CMIX	Yu et al., arXiv:2608.22862 (2026)
🚧	JANUS-AGE-MOEA-2-CMIX	Yu et al., arXiv:2608.22862 (2026)

Dominance & indicator-based MOEA — the NSGA-II lineage and its descendants.

	Method	Paper
✅	AGE-MOEA-2 (2022)	Panichella, GECCO 2022
✅	AGE-MOEA (2019)	Panichella, GECCO 2019
✅	NSGA-III (2014)	Deb & Jain, IEEE TEC 18(4), 2014
✅	SMS-EMOA (2007)	Beume, Naujoks & Emmerich, EJOR 181(3), 2007
✅	NSGA-II (2002)	Deb et al., IEEE TEC 6(2), 2002

Decomposition & reference-vector MOEA.

	Method	Paper
🚧	RVEA (2016)	Cheng et al., IEEE TEC 20(5), 2016
🚧	MOEA/D (2007)	Zhang & Li, IEEE TEC 11(6), 2007

CMA-ES / evolution-strategy family.

	Method	Paper
🚧	VkD-CMA (2016)	Akimoto & Hansen, GECCO 2016
🚧	LM-CMA (2015)	Loshchilov, arXiv:1511.00221 (2015)
🚧	CMA-ES (2001)	Hansen & Ostermeier, Evol. Comput. 9(2), 2001

Natural Evolution Strategies (NES) — natural-gradient search over a Gaussian distribution, from isotropic to full-covariance.

	Method	Paper
✅	OpenAI-ES (2017)	Salimans et al., arXiv:1703.03864 (2017)
✅	SNES (2014)	Wierstra et al., JMLR 15, 2014
✅	xNES (2014)	Wierstra et al., JMLR 15, 2014

Classic population-based & swarm.

	Method	Paper
🚧	Differential Evolution (1997)	Storn & Price, J. Global Optim. 11, 1997
🚧	Particle Swarm Optimization (1995)	Kennedy & Eberhart, Proc. IEEE ICNN 1995

Model-based (Bayesian / surrogate).

	Method	Paper
✅	TuRBO — Trust-Region Bayesian Optimization (2019)	Eriksson et al., NeurIPS 2019 (arXiv:1910.01739)
✅	REMBO — Random-Embedding Bayesian Optimization (2016)	Wang et al., JAIR 55, 2016 (doi:10.1613/jair.4806)
🚧	GP-BO — Gaussian-Process Bayesian Optimization (2012)	Snoek et al., NeurIPS 2012 (arXiv:1206.2944)
🚧	TPE — Tree-structured Parzen Estimator (2011)	Bergstra et al., NeurIPS 2011

Sampling & search baselines.

	Method	Paper
🚧	Random Search (2012)	Bergstra & Bengio, JMLR 13, 2012
🚧	QMC — Quasi-Monte Carlo (1967)	Sobol′, USSR Comput. Math. Math. Phys. 7(4), 1967
🚧	Grid Search	— (no canonical paper)
🚧	Brute-Force Search	— (no canonical paper)
🚧	Partial-Fixed	— (meta-sampler; no canonical paper)

Pruners — early-stopping methods

Bandit-based (successive-halving lineage).

	Method	Paper
🚧	Successive Halving / ASHA (2020)	Li et al., MLSys 2020 (arXiv:1810.05934)
🚧	Hyperband (2017)	Li et al., JMLR 18, 2017 (arXiv:1603.06560)

Statistical stopping rules.

	Method	Paper
🚧	Median (2017)	Golovin et al. (Google Vizier), KDD 2017
🚧	Wilcoxon (1945)	Wilcoxon, Biometrics Bulletin 1(6), 1945
🚧	Percentile	— (generalizes the median rule)
🚧	Threshold	— (outlier thresholding)
🚧	Patient	— (patience wrapper)
🚧	Nop	— (never prunes)

Importance evaluators — attribution methods

Density-based.

	Method	Paper
🚧	PED-ANOVA (2023)	Watanabe et al., IJCAI 2023 (arXiv:2304.10255)

Random-forest-surrogate based.

	Method	Paper
🚧	SHAP (2017)	Lundberg & Lee, NeurIPS 2017 (arXiv:1705.07874)
🚧	fANOVA (2014)	Hutter et al., ICML 2014
🚧	Mean Decrease Impurity (2001)	Breiman, Machine Learning 45(1), 2001

Terminators — stopping-criterion methods

GP-based regret.

	Method	Paper
🚧	EMMR — Expected Minimum Model Regret (2023)	Ishibashi et al., PMLR 206, 2023
🚧	Median Error Evaluator (2023)	Ishibashi et al., PMLR 206, 2023
🚧	Regret-Bound Evaluator (2022)	Makarova et al., AutoML-Conf 2022 (arXiv:2104.08166)

Utility evaluators.

	Method	Paper
🚧	Best-Value Stagnation	— (Optuna heuristic)
🚧	Cross-Validation Error	— (no canonical paper)
🚧	Static Error	— (no canonical paper)

Tier 2 — building blocks

Operators — recombination & mutation

SBX family (simulated binary).

	Operator	Paper
🚧	vSBX — Modified Simulated Binary Crossover (2002)	Modified SBX, PPSN VII (LNCS 2439), 2002
🚧	SBX — Simulated Binary Crossover (1995)	Deb & Agrawal, Complex Systems 9(2), 1995

Simplex & normal-distribution recombination (multi-parent).

	Operator	Paper
🚧	SPX — Simplex Crossover (2001)	Tsutsui, Goldberg & Sastry, 2001
🚧	UNDX — Unimodal Normal Distribution Crossover (1999)	Kita, Ono & Kobayashi, CEC 1999

Blend & discrete recombination.

	Operator	Paper
🚧	BLX-α — Blend Crossover (1993)	Eshelman & Schaffer, FOGA 1993
🚧	Uniform Crossover (1989)	Syswerda, ICGA 1989

Mutation.

	Operator	Paper
🚧	Polynomial Mutation (1996)	Deb & Goyal (GeneAS), 1996

Kernels — numerical primitives

	Kernel	Paper
🚧	GP Acquisition Functions (2023)	Ament et al. (LogEI), NeurIPS 2023 (arXiv:2310.20708)
🚧	Hypervolume (WFG) (2012)	While, Bradstreet & Barone, IEEE TEC 16(1), 2012
🚧	CMA Optimizers (2016)	Hansen, The CMA Evolution Strategy: A Tutorial (arXiv:1604.00772)
🚧	Reference Directions — Das–Dennis (1998)	Das & Dennis, SIAM J. Optim. 8(3), 1998
🚧	QMC Sequences — Sobol′ / Halton (1967)	Sobol′, USSR Comput. Math. Math. Phys. 7(4), 1967

Via integration bridges

These methods are driven through a foreign runtime and live in optional satellite packages, so they are documented with their integrations (see the Samplers guide) rather than as native deep-dives:

BoTorch — BoTorchSampler (OptunaSharp.Interop.BoTorch): Bayesian optimization via the real BoTorch (Balandat et al., NeurIPS 2020).
PyCma — PyCmaSampler (OptunaSharp.Interop.PyCma): CMA-ES via Hansen's cma package.
Pymoo — PymooSampler (OptunaSharp.Interop.Pymoo): drives 11 pymoo multi-objective algorithms (NSGA-II/III, R-NSGA-II/III, U-NSGA-III, AGE-MOEA/2, RVEA, SMS-EMOA, SPEA2, C-TAEA).
OptunaHub — OptunaHubSampler / OptunaHubPruner (OptunaSharp.Interop.pythonnet): wraps any of the ~85 samplers and the pruners on hub.optuna.org.
Forest-backend importances — ML.NET FastForest/FastTree and LightGBM variants of fANOVA / MDI (OptunaSharp.Integration.ML.NET, OptunaSharp.Integration.LightGbm).

Table of Contents