Tutorial - How to Compare Evolutionary Multi-Objective Optimization Algorithms: Parameter Specifications, Indicators and Test Problems

Lie Meng Pang,Southern University of Science and Technology, China; Hisao Ishibuchi, Southern University of Science and Technology, China; Lie Meng Pang, Southern University of Science and Technology, China; Ke Shang, Southern University of Science and Technology, China ABSTRACT:Evolutionary multi-objective optimization (EMO) has been a very active research area in recent years. Almost every year, new EMO algorithms are proposed. When a new EMO algorithm is proposed, computational experiments are usually conducted in order to compare its performance with existing algorithms. Then, experimental results are summarized and reported as a number of tables together with statistical significance test results. Those results usually show higher performance of the new algorithm than existing algorithms. However, fair comparison of different EMO algorithms is not easy since the evaluated performance of each algorithm usually depends on experimental settings. This is also because solution sets instead of solutions are evaluated. In this tutorial, we will first explain some commonly-used software platforms and experimental settings for the comparison of EMO algorithms. Then, we will discuss how to specify the common setting of computational experiments, which is used by all the compared EMO algorithms. More specifically, the focus of this tutorial is the setting related to the following four issues: (i) termination condition, (ii) population size, (iii) performance indicators, (iv) test problem. For each issue, we will provide a clear demonstration of its strong effects on comparison results of EMO algorithms. Following that, we will discuss how to handle each of these issues for fair comparison. These discussions aim to encourage the future development of the EMO research field without focusing too much on the development of overly-specialized new algorithms in a specific setting. Finally, we will also suggest some promising future research topics related to each issue.