Ontogeny Recapitulates Phylogeny: Evolutionary Insights into Hyperparameter Tuning

Abstract

Recent work has suggested that the morphological development of feedforward neural networks, which perform various complex tasks, resembles evolutionary adaptations in individual species. This paper investigates if the development of explicitly ontogenetic feedforward neural networks mimics the adaptation processes following 'ontogeny' inductions from many different species. After growing these neural networks from embryos, we found that recognizing a robot's behavioral objectives during a task, which is akin to identifying task load demands, turned out to be associated with hyperparameter tuning and morphological coding, as in evolution. We conjecture that neural network ontogeny captures insights into a recurrent biological dichotomy, where one major evolutionary question is how diversity arises, and this is juxtaposed with the classical axiomatic argument in genetics that highly canalized traits lead to organisms with high values of Shannon mutual entropy functioning properly. Based on these findings, it is evident that the remarkable similarities between neural network development and evolutionary processes extend beyond mere resemblances, reinforcing the hypothesis that ontogenetic feedforward neural networks not only resemble evolutionary adaptations, but also actively parallel them in their quest for optimal functionality and adaptation to a wide spectrum of environmental demands.