Tutorial - Benchmarking and Analyzing Iterative Optimization Heuristics with IOHprofiler

Diederick Vermetten,Leiden University; Furong Ye, Leiden University, Jacob de Nobel, Leiden University, Hao Wang, Leiden University, Carola Doerr, Sorbonne University, France; Thomas Bäck, Leiden University ABSTRACT: Evaluating sampling-based optimization algorithms via sound benchmarking is a core concern in evolutionary computation. IOHprofiler supports researchers in this task by providing an easy-to-use, interactive, and highly customizable environment for benchmarking any iterative optimizer. The experimenter module provides easy access to common problem sets (e.g. BBOB functions) and modular logging functionality that can be easily combined with other optimization functions. The resulting logs (and logs from other platforms, e.g. COCO and Nevergrad) are fully interoperable with the IOHanalyzer, which provides access to highly interactive performance analysis, in the form of a wide array of visualizations and statistical analyses. A GUI, hosted at https://iohanalyzer.liacs.nl/ makes these analysis tools easy to access. Data from many repositories (e.g. COCO, Nevergrad) are pre-processed, such that the effort required to compare performance to existing algorithms is greatly reduced. This tutorial will introduce the motivation of the IOHprofiler project and its core functionality. The key features and components will be highlighted and demonstrated by the organizers. Afterward, there will be a hands-on session where the usage of the various analysis functionalities is demonstrated. Guided examples will be provided to highlight the many aspects of algorithm performance which can be explored using the interactive GUI. For this session, no software installation is necessary.