Conference on Genetic and Evolutionary Computation 2022

A Discretization-Free Metric For Assessing Quality Diversity Algorithms

LEFT – Figure 2: A visualisation of the changing optimal solution as the weighting changes // RIGHT – Figure 6: A visualisation showing the Pareto Front of a solution set from MAP-Elites on the Rosenbrock6d problem with a 10×10 archive and distances measured from a single target point. Orange crosses indicate dominated solutions, blue area shows the hypervolume

Abstract

A Discretization-free Metric For Assessing Quality Diversity Algorithms

Paul Kent, Juergen Branke, Adam Gaier, Jean-Baptiste Mouret

Annual Conference Companion on Genetic and Evolutionary Computation 2022

While Quality-Diversity algorithms attempt to produce a set of high quality solutions that are diverse throughout descriptor space, in reality decision makers are often interested in solutions with specific descriptor values. In this paper we suggest that current methods of evaluating Quality Diversity algorithm performance do not properly account for a decision maker’s preference in a continuous descriptor space and suggest three approaches that attempt to capture the real-world trade-off between a solution’s objective performance and distance from a desired set of target descriptors. In this paper we propose a randomised metric, a process of Monte-Carlo sampling of $n$ target points in descriptor space and a small number of random weights that represent different tolerances for mis-specification in a solution’s descriptor values. This sampling allows us to simulate the requirements of all possible combinations of target-tolerance pairs and, by taking sufficient samples, estimate average performance. We go on to formulate three simple methods for comparing average performance of algorithms; Continuous Quality Diversity score (CQD) and Hypervolume of the objective/distance Pareto front. We show that these measures are simple to implement and robust measures of performance without introducing artificial discretisation of the descriptor space.

Download publication

Associated Researchers

Paul Kent

Warwick University

Juergen Branke

Warwick Business School

Jean-Baptiste Mouret

Inria, CNRS, Université de Lorraine

View all researchers

Related Resources

Publication

2023

Extracting Design Knowledge from Optimization Data: Enhancing Engineering Design in Fluid Based Thermal Management Systems

Extracting knowledge from optimization data in multi-split thermal…

Publication

2021

Building-GAN: Graph-Conditioned Architectural Volumetric Design Generation

Volumetric design is the first and critical step for professional…

Publication

2021

Robust Representation Learning via Perceptual Similarity Metrics

A fundamental challenge in artificial intelligence is learning useful…

Publication

2020

Contrastive Multi-View Representation Learning on Graphs

We introduce a self-supervised approach for learning node and graph…

Get in touch

Something pique your interest? Get in touch if you’d like to learn more about Autodesk Research, our projects, people, and potential collaboration opportunities.

Contact us