A Quality-Diversity-based Evaluation Strategy for Symbolic Music Generation

Since human (audience) evaluation methods are challenging due to their logistic inconvenience, symbolic music generation systems are typically evaluated using loss-based measures or some statistical analyses with pre-defined musical metrics. Even though these loss-based and statistical methods could be informative to some extent, they often cannot guarantee any success for the generative model in terms of higher-level musical qualities, such as style / genre. Also, as another aspect of evaluation, diversity of the generated material is not considered for symbolic music generators. In this study, we argue that Quality-Diversity-based evaluation approach is more appropriate to value symbolic music generators. We give a few examples of where loss-based and statistical methods fail and suggest some techniques for quality-based and diversity-based evaluation, jointly forming a Quality-Diversity-based evaluation strategy.