Conference on Genetic and Evolutionary Computation 2022
A Discretization-Free Metric For Assessing Quality Diversity Algorithms
Abstract
A Discretization-free Metric For Assessing Quality Diversity Algorithms
Paul Kent, Juergen Branke, Adam Gaier, Jean-Baptiste Mouret
Annual Conference Companion on Genetic and Evolutionary Computation 2022
While Quality-Diversity algorithms attempt to produce a set of high quality solutions that are diverse throughout descriptor space, in reality decision makers are often interested in solutions with specific descriptor values. In this paper we suggest that current methods of evaluating Quality Diversity algorithm performance do not properly account for a decision maker’s preference in a continuous descriptor space and suggest three approaches that attempt to capture the real-world trade-off between a solution’s objective performance and distance from a desired set of target descriptors. In this paper we propose a randomised metric, a process of Monte-Carlo sampling of $n$ target points in descriptor space and a small number of random weights that represent different tolerances for mis-specification in a solution’s descriptor values. This sampling allows us to simulate the requirements of all possible combinations of target-tolerance pairs and, by taking sufficient samples, estimate average performance. We go on to formulate three simple methods for comparing average performance of algorithms; Continuous Quality Diversity score (CQD) and Hypervolume of the objective/distance Pareto front. We show that these measures are simple to implement and robust measures of performance without introducing artificial discretisation of the descriptor space.
Download publicationAssociated Researchers
Paul Kent
Warwick University
Juergen Branke
Warwick Business School
Jean-Baptiste Mouret
Inria, CNRS, Université de Lorraine
Related Resources
2024
Optimal design of frame structures with mixed categorical and continuous design variables using the Gumbel–Softmax method
New gradient-based optimizer for handling budget and material…
2023
SolidGen: An Autoregressive Model for Direct B-rep Synthesis
A generative model that can synthesize 3D CAD models in the boundary…
2023
3DALL-E: Integrating Text-to-Image AI in 3D Design Workflows
3DALL-E integrated three large AI models within Fusion 360 to explore…
2022
JoinABLe: Learning Bottom-up Assembly of Parametric CAD Joints
Physical products are often complex assemblies combining a multitude…
Get in touch
Something pique your interest? Get in touch if you’d like to learn more about Autodesk Research, our projects, people, and potential collaboration opportunities.
Contact us