Title:Consecutive Preferential Bayesian Optimization
Abstract:Preferential Bayesian optimization allows optimization of objectives that are either expensive or difficult to measure directly, by relying on a minimal number of comparative evaluations done by a human expert. Generating candidate solutions for evaluation is also often expensive, but this cost is ignored by existing methods. We generalize preference-based optimization to explicitly account for production and evaluation costs with Consecutive Preferential Bayesian Optimization, reducing production cost by constraining comparisons to involve previously generated candidates. We also account for the perceptual ambigui…
Title:Consecutive Preferential Bayesian Optimization
Abstract:Preferential Bayesian optimization allows optimization of objectives that are either expensive or difficult to measure directly, by relying on a minimal number of comparative evaluations done by a human expert. Generating candidate solutions for evaluation is also often expensive, but this cost is ignored by existing methods. We generalize preference-based optimization to explicitly account for production and evaluation costs with Consecutive Preferential Bayesian Optimization, reducing production cost by constraining comparisons to involve previously generated candidates. We also account for the perceptual ambiguity of the oracle providing the feedback by incorporating a Just-Noticeable Difference threshold into a probabilistic preference model to capture indifference to small utility differences. We adapt an information-theoretic acquisition strategy to this setting, selecting new configurations that are most informative about the unknown optimum under a preference model accounting for the perceptual ambiguity. We empirically demonstrate a notable increase in accuracy in setups with high production costs or with indifference feedback.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2511.05163 [cs.LG] |
| (or arXiv:2511.05163v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2511.05163 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Aras Erarslan [view email] [v1] Fri, 7 Nov 2025 11:30:36 UTC (16,442 KB)