doi:10.1017/S0269888921000151

Aerts , H., Fias , W., Caeyenberghs , K. & Marinazzo , D. (2016). Brain networks under attack: robustness properties and the impact of lesions. Brain 139(12), 3063–3083.

Ansuini , A., Medvet , E., Pellegrino , F. A. & Zullich , M. (2020a). Investigating similarity metrics for convolutional neural networks in the case of unstructured pruning. In International Conference on Pattern Recognition Applications and Methods, 87–111, Springer.

Ansuini , A., Medvet , E., Pellegrino , F. A. & Zullich , M. (2020b). On the similarity between hidden layers of pruned and unpruned convolutional neural networks. In ICPRAM, 52–59.

Anwar , S., Hwang , K. & Sung , W. (2017). Structured pruning of deep convolutional neural networks. ACM Journal on Emerging Technologies in Computing Systems (JETC) 13(3), 1–18.

Arcuri , C., Mecca , C., Bianchi , R., Giambanco , I. & Donato , R. (2017). The pathophysiological role of microglia in dynamic surveillance, phagocytosis and structural remodeling of the developing cns. Frontiers in Molecular Neuroscience 10, 191.

Bartoldson , B. R., Morcos , A. S., Barbu , A. & Erlebacher , G. (2019). The generalization-stability tradeoff in neural network pruning. arXiv preprint arXiv:1906.03728.

Bassett , D. S. & Sporns , O. (2017). Network Neuroscience Nature Neuroscience 20(3), 353–364.

Bengio , Y., Le Roux , N., Vincent , P., Delalleau , O. & Marcotte , P. (2006). Convex neural networks. In Advances in Neural Information Processing systems, 18, 123.

Bishop , C. M. (1995). Neural Networks for Pattern Recognition. Clarendon Press, chapter 9.5.3 - Saliency of Weights.

Bongard , J. C. (2011). Morphological and environmental scaffolding synergize when evolving robot controllers: artificial life/robotics/evolvable hardware, 179–186.

Bordier , C., Nicolini , C. & Bifone , A. (2017). Graph analysis and modularity of brain functional connectivity networks: searching for the optimal threshold. Frontiers in Neuroscience 11, 441.

Breiman , L., Friedman , J., Stone , C. J. & Olshen , R. A. (1984). Classification and Regression Trees. CRC Press.

Brown , T., Mann , B., Ryder , N., Subbiah , M., Kaplan , J. D., Dhariwal , P., Neelakantan , A., Shyam , P., Sastry , G., Askell , A., Agarwal , S., Herbert-Voss , A., Krueger , G., Henighan , T., Child , R., Ramesh , A., Ziegler , D., Wu , J., Winter , C., Hesse , C., Chen , M., Sigler , E., Litwin , M., Gray , S., Chess , B., Clark , J., Berner , C., McCandlish , S., Radford , A., Sutskever , I. & Amodei , D. (2020). Language models are few-shot learners. In Advances in Neural Information Processing Systems, Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M. F. & Lin, H. (eds), 33. Curran Associates, Inc., 1877–1901. https://proceedings.neurips.cc/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf.

Cai , L., An , Z., Yang , C. & Xu , Y. (2021). Softer pruning, incremental regularization. In 2020 25th International Conference on Pattern Recognition (ICPR), 224–230.

Cheney , N., Bongard , J. & Lipson , H. (2015). Evolving soft robots in tight spaces. In Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation, 935–942, ACM.

Cheney , N., Clune , J. & Lipson , H. (2014). Evolved electrophysiological soft robots. In Artificial Life Conference Proceedings 14. MIT Press, 222–229.

Denève , S., Alemi , A. & Bourdoukan , R. (2017). The brain as an efficient and robust adaptive learner. Neuron 94(5), 969–977.

Elbrecht , D. & Schuman , C. (2020). Neuroevolution of spiking neural networks using compositional pattern producing networks. In International Conference on Neuromorphic Systems 2020, 1–5.

Fedus , W., Zoph , B. & Shazeer , N. (2021). Switch transformers: scaling to trillion parameter models with simple and efficient sparsity. arXiv preprint arXiv:2101.03961.

Floreano , D., Dürr , P. & Mattiussi , C. (2008). Neuroevolution: from architectures to learning. Evolutionary Intelligence 1(1), 47–62.

Frankle , J. & Carbin , M. (2019). The lottery ticket hypothesis: finding sparse, trainable neural networks. In International Conference on Learning Representations. https://openreview.net/forum?id=rJl-b3RcF7.

Gerstner , W. & Kistler , W. M. (2002). Spiking Neuron Models: Single Neurons, Populations, Plasticity. Cambridge University Press.

Gerum , R. C., Erpenbeck , A., Krauss , P. & Schilling , A. (2020). Sparsity through evolutionary pruning prevents neuronal networks from overfitting. Neural Networks 128, 305–312.

Han , S., Pool , J., Tran , J. & Dally , W. (2015). Learning both weights and connections for efficient neural network. In Advances in Neural Information Processing Systems 28, Cortes , C., Lawrence , N. D., Lee , D. D., Sugiyama , M. & Garnett , R. (eds). Curran Associates, Inc., 1135–1143.

Hebb , D. O. (2005). The Organization of Behavior: A Neuropsychological Theory. Psychology Press.

Heiney , K., Huse Ramstad , O., Fiskum , V., Christiansen , N., Sandvig , A., Nichele , S. & Sandvig , I. (2021). Criticality, connectivity, and neural disorder: a multifaceted approach to neural computation. Frontiers in Computational Neuroscience15, 7.

Herculano-Houzel , S. (2012). The remarkable, yet not extraordinary, human brain as a scaled-up primate brain and its associated cost. Proceedings of the National Academy of Sciences 109(Supplement 1), 10661–10668.

Hiller , J. & Lipson , H. (2012). Automatic design and manufacture of soft robots. IEEE Transactions on Robotics 28(2), 457–466.

Hoefler , T., Alistarh , D., Ben-Nun , T., Dryden , N. & Peste , A. (2021). Sparsity in deep learning: pruning and growth for efficient inference and training in neural networks. arXiv preprint arXiv:2102.00554.

Horibe , K., Walker , K. & Risi , S. (2021). Regenerating soft robots through neural cellular automata. In EuroGP, 36–50.

Iglesias , J., Eriksson , J., Grize , F., Tomassini , M. & Villa , A. E. (2005). Dynamics of pruning in simulated large-scale spiking neural networks. Biosystems 79(1–3), 11–20.

Johnson , M. H. (2001). Functional brain development in humans. Nature Reviews Neuroscience 2(7), 475–483.

Kassahun , Y. & Sommer , G. (2005). Efficient reinforcement learning through evolutionary acquisition of neural topologies. In ESANN, 259–266.

Kriegman , S., Cheney , N. & Bongard , J. (2018). How morphological development can guide evolution. Scientific Reports 8(1), 13934.

Kriegman , S., Cheney , N., Corucci , F. & Bongard , J. C. (2018). Interoceptive robustness through environment-mediated morphological development. arXiv preprint arXiv:1804.02257.

Laughlin , S. B., van Steveninck , R. R. d. R. & Anderson , J. C. (1998). The metabolic cost of neural information. Nature Neuroscience 1(1), 36–41.

Laurenti , L., Patane , A., Wicker , M., Bortolussi , L., Cardelli , L. & Kwiatkowska , M. (2019). Global adversarial robustness guarantees for neural networks.

LeCun , Y., Denker , J. S., Solla , S. A., Howard , R. E. & Jackel , L. D. (1989). Optimal brain damage. In NIPs, 2. Citeseer, 598–605.

Liao , X., Vasilakos , A. V. & He , Y. (2017). Small-world human brain networks: perspectives and challenges. Neuroscience & Biobehavioral Reviews 77, 286–300.

Lin , T., Stich , S. U., Barba , L., Dmitriev , D. & Jaggi , M. (2020). Dynamic model pruning with feedback. In International Conference on Learning Representations. https://openreview.net/forum?id=SJem8lSFwB.

Lipson , H., Sunspiral , V., Bongard , J. & Cheney , N. (2016). On the difficulty of co-optimizing morphology and control in evolved virtual creatures. In Artificial Life Conference Proceedings 13. MIT Press, 226–233.

Liu , S., Chen , T., Chen , X., Atashgahi , Z., Yin , L., Kou , H., Shen , L., Pechenizkiy , M., Wang , Z. & Mocanu , D. C. (2021). Sparse training via boosting pruning plasticity with neuroregeneration. arXiv preprint arXiv:2106.10404.

Liu , Z., Sun , M., Zhou , T., Huang , G. & Darrell , T. (2019). Rethinking the value of network pruning. In International Conference on Learning Representations. https://openreview.net/forum?id=rJlnB3C5Ym.

Low , L. K. & Cheng , H.-J. (2006). Axon pruning: an essential step underlying the developmental plasticity of neuronal connections. Philosophical Transactions of the Royal Society B: Biological Sciences 361(1473), 1531–1544.

Medvet , E., Bartoli , A., De Lorenzo , A. & Fidel , G. (2020). Evolution of distributed neural controllers for voxel-based soft robots. In Proceedings of the 2020 Genetic and Evolutionary Computation Conference, 112–120.

Medvet , E., Bartoli , A., De Lorenzo , A. & Seriani , S. (2020). 2D-VSR-Sim: a simulation tool for the optimization of 2-D voxel-based soft robots. SoftwareX 12.

Medvet , E., Bartoli , A., Pigozzi , F. & Rochelli , M. (2021). Biodiversity in evolved voxel-based soft robots. In Proceedings of the Genetic and Evolutionary Computation Conference, 129–137.

Meunier , D., Lambiotte , R. & Bullmore , E. T. (2010). Modular and hierarchically modular organization of brain networks. Frontiers in Neuroscience 4, 200.

Mordvintsev , A., Randazzo , E., Niklasson , E. & Levin , M. (2020). Growing neural cellular automata. Distill 5(2), e23.

Nadizar , G., Medvet , E., Pellegrino , F. A., Zullich , M. & Nichele , S. (2021). On the effects of pruning on evolved neural controllers for soft robots. In Proceedings of the Genetic and Evolutionary Computation Conference Companion, 1744–1752.

Naumov , M., Chien , L., Vandermersch , P. & Kapasi , U. (2010). Cusparse library. In GPU Technology Conference.

Neyshabur , B., Li , Z., Bhojanapalli , S., LeCun , Y. & Srebro , N. (2019). The role of over-parametrization in generalization of neural networks. In International Conference on Learning Representations. https://openreview.net/forum?id=BygfghAcYX.

Nichele , S., Ose , M. B., Risi , S. & Tufte , G. (2017). Ca-neat: evolved compositional pattern producing networks for cellular automata morphogenesis and replication. IEEE Transactions on Cognitive and Developmental Systems 10(3), 687–700.

Niklasson , E., Mordvintsev , A., Randazzo , E. & Levin , M. (2021). Self-organising textures. Distill 6(2), e00027–003.

Pfeifer , R. & Bongard , J. (2006). How the Body Shapes the Way We Think: A New View of Intelligence. MIT Press.

Pontes-Filho , S. & Nichele , S. (2019). Towards a framework for the evolution of artificial general intelligence. arXiv preprint arXiv:1903.10410.

Power , J. D. & Schlaggar , B. L. (2017). Neural plasticity across the lifespan. Wiley Interdisciplinary Reviews: Developmental Biology 6(1), e216.

Prakash , A., Storer , J., Florencio , D. & Zhang , C. (2019). Repr: Improved training of convolutional filters. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 10666–10675.

Qiu , H., Garratt , M., Howard , D. & Anavatti , S. (2018). Evolving spiking neural networks for nonlinear control problems. In 2018 IEEE Symposium Series on Computational Intelligence (SSCI), 1367–1373, IEEE.

Raman , D. V., Rotondo , A. P. & O’Leary , T. (2019). Fundamental bounds on learning performance in neural circuits. Proceedings of the National Academy of Sciences 116(21), 10537–10546.

Ramesh , A., Pavlov , M., Goh , G., Gray , S., Voss , C., Radford , A., Chen , M. & Sutskever , I. (2021). Zero-shot text-to-image generation. arXiv preprint arXiv:2102.12092.

Real , E., Moore , S., Selle , A., Saxena , S., Suematsu , Y. L., Tan , J., Le , Q. V. & Kurakin , A. (2017). Large-scale evolution of image classifiers. In International Conference on Machine Learning, 2902–2911. PMLR.

Renda , A., Frankle , J. & Carbin , M. (2020). Comparing fine-tuning and rewinding in neural network pruning. In International Conference on Learning Representations.

Riccomagno , M. M. & Kolodkin , A. L. (2015). Sculpting neural circuits by axon and dendrite pruning. Annual Review of Cell and Developmental Biology 31, 779–805.

Sakai , J. (2020). Core concept: how synaptic pruning shapes neural wiring during development and, possibly, in disease. Proceedings of the National Academy of Sciences 117(28), 16096–16099.

Santosa , F. & Symes , W. W. (1986). Linear inversion of band-limited reflection seismograms. SIAM Journal on Scientific and Statistical Computing 7(4), 1307–1330.

Schuldiner , O. & Yaron , A. (2015). Mechanisms of developmental neurite pruning. Cellular and Molecular Life Sciences 72(1), 101–119.

Segev , I. & Schneidman , E. (1999). Axons as computing devices: basic insights gained from models. Journal of Physiology-Paris 93(4), 263–270.

Shi , Y., Nguyen , L., Oh , S., Liu , X. & Kuzum , D. (2019). A soft-pruning method applied during training of spiking neural networks for in-memory computing applications. Frontiers in Neuroscience 13, 405.

Siebel , N. T., Botel , J. & Sommer , G. (2009). Efficient neural network pruning during neuro-evolution. In 2009 International Joint Conference on Neural Networks, 2920–2927, IEEE.

Sporns , O. (2013). Structure and function of complex brain networks. Dialogues in Clinical Neuroscience 15(3), 247.

Sporns , O., Chialvo , D. R., Kaiser , M. & Hilgetag , C. C. (2004). Complex networks: small-world and scale-free architectures. Trends in Cognitive Sciences 9(8), 418–425.

Stanley , K. O. & Miikkulainen , R. (2002). Evolving neural networks through augmenting topologies. Evolutionary Computation 10(2), 99–127.

Strubell , E., Ganesh , A. & McCallum , A. (2019). Energy and policy considerations for deep learning in nlp. arXiv preprint arXiv:1906.02243.

Sudhakaran , S., Grbic , D., Li , S., Katona , A., Najarro , E., Glanois , C. & Risi , S. (2021). Growing 3D artefacts and functional machines with neural cellular automata. arXiv preprint arXiv:2103.08737.

Talamini , J., Medvet , E., Bartoli , A. & De Lorenzo , A. (2019). Evolutionary synthesis of sensing controllers for voxel-based soft robots. In Artificial Life Conference Proceedings, MIT Press, 574–581.

Thimm , G. & Fiesler , E. (1995). Evaluating pruning methods. In Proceedings of the International Symposium on Artificial neural networks, 20–25, Citeseer.

Thodberg , H. H. (1991). Improving generalization of neural networks through pruning. International Journal of Neural Systems 1(04), 317–326.

Tibshirani , R. (1997). The lasso method for variable selection in the cox model. Statistics in medicine 16(4), 385–395.

Variengien , A., Nichele , S., Glover , T. & Pontes-Filho , S. (2021). Towards self-organized control: Using neural cellular automata to robustly control a cart-pole agent. arXiv preprint arXiv:2106.15240.

Vézquez-Rodrguez , B., Liu , Z.-Q., Hagmann , P. & Misic , B. (2020). Signal propagation via cortical hierarchies. Network Neuroscience 4(4), 1072–1090.

Ye , S., Xu , K., Liu , S., Cheng , H., Lambrechts , J.-H., Zhang , H., Zhou , A., Ma , K., Wang , Y. & Lin , X. (2019). Adversarial robustness vs. model compression, or both?. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 111–120.

You , H., Li , C., Xu , P., Fu , Y., Wang , Y., Chen , X., Baraniuk , R. G., Wang , Z. & Lin , Y. (2019). Drawing early-bird tickets: Towards more efficient training of deep networks. arXiv preprint arXiv:1909.11957.

Yuste , R. (2015). From the neuron doctrine to neural networks. Nature Reviews Neuroscience 16(8), 487–497.

Zhang , B.-T. & Mühlenbein , H. (1993). Genetic programming of minimal neural nets using occam’s razor. In Proceedings of the 5th International Conference on Genetic Algorithms (ICGA’93). Citeseer.

Zullich , M., Medvet , E., Pellegrino , F. A. & Ansuini , A. (2021). Speeding-up pruning for artificial neural networks: introducing accelerated iterative magnitude pruning. In 2020 25th International Conference on Pattern Recognition (ICPR), 3868–3875. IEEE.