Department of Engineering and Architecture, University of Trieste, Italy Email: giorgia.nadizar@phd.units.it, emedvet@units.it, fapellegrino@units.it, marco.zullich@phd.units.it"/> Department of Computer Science, Artificial Intelligence Lab, Oslo Metropolitan University, Norway"/> Department of Neuromedicine and Movement Science, Norwegian University of Science and Technology, Norway Email: olahuser@oslomet.no"/> Department of Holistic Systems, Simula Metropolitan Center for Digital Engineering, Norway Email: stenic@oslomet.no"/>
Search
2022 Volume 37
Article Contents
RESEARCH ARTICLE   Open Access    

Merging pruning and neuroevolution: towards robust and efficient controllers for modular soft robots

  • The online version of this article has been updated since its original publication. A notice detailing the changes has been published at: https://doi.org/10.1017/S0269888922000017

More Information
  • Abstract: Artificial neural networks (ANNs) can be employed as controllers for robotic agents. Their structure is often complex, with many neurons and connections, especially when the robots have many sensors and actuators distributed across their bodies and/or when high expressive power is desirable. Pruning (removing neurons or connections) reduces the complexity of the ANN, thus increasing its energy efficiency, and has been reported to improve the generalization capability, in some cases. In addition, it is well-known that pruning in biological neural networks plays a fundamental role in the development of brains and their ability to learn. In this study, we consider the evolutionary optimization of neural controllers for the case study of Voxel-based soft robots, a kind of modular, bio-inspired soft robots, applying pruning during fitness evaluation. For a locomotion task, and for centralized as well as distributed controllers, we experimentally characterize the effect of different forms of pruning on after-pruning effectiveness, life-long effectiveness, adaptability to new terrains, and behavior. We find that incorporating some forms of pruning in neuroevolution leads to almost equally effective controllers as those evolved without pruning, with the benefit of higher robustness to pruning. We also observe occasional improvements in generalization ability.
  • 加载中
  • Aerts , H., Fias , W., Caeyenberghs , K. & Marinazzo , D. (2016). Brain networks under attack: robustness properties and the impact of lesions. Brain 139(12), 3063–3083.

    Google Scholar

    Ansuini , A., Medvet , E., Pellegrino , F. A. & Zullich , M. (2020a). Investigating similarity metrics for convolutional neural networks in the case of unstructured pruning. In International Conference on Pattern Recognition Applications and Methods, 87–111, Springer.

    Google Scholar

    Ansuini , A., Medvet , E., Pellegrino , F. A. & Zullich , M. (2020b). On the similarity between hidden layers of pruned and unpruned convolutional neural networks. In ICPRAM, 52–59.

    Google Scholar

    Anwar , S., Hwang , K. & Sung , W. (2017). Structured pruning of deep convolutional neural networks. ACM Journal on Emerging Technologies in Computing Systems (JETC) 13(3), 1–18.

    Google Scholar

    Arcuri , C., Mecca , C., Bianchi , R., Giambanco , I. & Donato , R. (2017). The pathophysiological role of microglia in dynamic surveillance, phagocytosis and structural remodeling of the developing cns. Frontiers in Molecular Neuroscience 10, 191.

    Google Scholar

    Bartoldson , B. R., Morcos , A. S., Barbu , A. & Erlebacher , G. (2019). The generalization-stability tradeoff in neural network pruning. arXiv preprint arXiv:1906.03728.

    Google Scholar

    Bassett , D. S. & Sporns , O. (2017). Network Neuroscience Nature Neuroscience 20(3), 353–364.

    Google Scholar

    Bengio , Y., Le Roux , N., Vincent , P., Delalleau , O. & Marcotte , P. (2006). Convex neural networks. In Advances in Neural Information Processing systems, 18, 123.

    Google Scholar

    Bishop , C. M. (1995). Neural Networks for Pattern Recognition. Clarendon Press, chapter 9.5.3 - Saliency of Weights.

    Google Scholar

    Bongard , J. C. (2011). Morphological and environmental scaffolding synergize when evolving robot controllers: artificial life/robotics/evolvable hardware, 179–186.

    Google Scholar

    Bordier , C., Nicolini , C. & Bifone , A. (2017). Graph analysis and modularity of brain functional connectivity networks: searching for the optimal threshold. Frontiers in Neuroscience 11, 441.

    Google Scholar

    Breiman , L., Friedman , J., Stone , C. J. & Olshen , R. A. (1984). Classification and Regression Trees. CRC Press.

    Google Scholar

    Brown , T., Mann , B., Ryder , N., Subbiah , M., Kaplan , J. D., Dhariwal , P., Neelakantan , A., Shyam , P., Sastry , G., Askell , A., Agarwal , S., Herbert-Voss , A., Krueger , G., Henighan , T., Child , R., Ramesh , A., Ziegler , D., Wu , J., Winter , C., Hesse , C., Chen , M., Sigler , E., Litwin , M., Gray , S., Chess , B., Clark , J., Berner , C., McCandlish , S., Radford , A., Sutskever , I. & Amodei , D. (2020). Language models are few-shot learners. In Advances in Neural Information Processing Systems, Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M. F. & Lin, H. (eds), 33. Curran Associates, Inc., 1877–1901. https://proceedings.neurips.cc/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf.

    Google Scholar

    Cai , L., An , Z., Yang , C. & Xu , Y. (2021). Softer pruning, incremental regularization. In 2020 25th International Conference on Pattern Recognition (ICPR), 224–230.

    Google Scholar

    Cheney , N., Bongard , J. & Lipson , H. (2015). Evolving soft robots in tight spaces. In Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation, 935–942, ACM.

    Google Scholar

    Cheney , N., Clune , J. & Lipson , H. (2014). Evolved electrophysiological soft robots. In Artificial Life Conference Proceedings 14. MIT Press, 222–229.

    Google Scholar

    Denève , S., Alemi , A. & Bourdoukan , R. (2017). The brain as an efficient and robust adaptive learner. Neuron 94(5), 969–977.

    Google Scholar

    Elbrecht , D. & Schuman , C. (2020). Neuroevolution of spiking neural networks using compositional pattern producing networks. In International Conference on Neuromorphic Systems 2020, 1–5.

    Google Scholar

    Fedus , W., Zoph , B. & Shazeer , N. (2021). Switch transformers: scaling to trillion parameter models with simple and efficient sparsity. arXiv preprint arXiv:2101.03961.

    Google Scholar

    Floreano , D., Dürr , P. & Mattiussi , C. (2008). Neuroevolution: from architectures to learning. Evolutionary Intelligence 1(1), 47–62.

    Google Scholar

    Frankle , J. & Carbin , M. (2019). The lottery ticket hypothesis: finding sparse, trainable neural networks. In International Conference on Learning Representations. https://openreview.net/forum?id=rJl-b3RcF7.

    Google Scholar

    Gerstner , W. & Kistler , W. M. (2002). Spiking Neuron Models: Single Neurons, Populations, Plasticity. Cambridge University Press.

    Google Scholar

    Gerum , R. C., Erpenbeck , A., Krauss , P. & Schilling , A. (2020). Sparsity through evolutionary pruning prevents neuronal networks from overfitting. Neural Networks 128, 305–312.

    Google Scholar

    Han , S., Pool , J., Tran , J. & Dally , W. (2015). Learning both weights and connections for efficient neural network. In Advances in Neural Information Processing Systems 28, Cortes , C., Lawrence , N. D., Lee , D. D., Sugiyama , M. & Garnett , R. (eds). Curran Associates, Inc., 1135–1143.

    Google Scholar

    Hebb , D. O. (2005). The Organization of Behavior: A Neuropsychological Theory. Psychology Press.

    Google Scholar

    Heiney , K., Huse Ramstad , O., Fiskum , V., Christiansen , N., Sandvig , A., Nichele , S. & Sandvig , I. (2021). Criticality, connectivity, and neural disorder: a multifaceted approach to neural computation. Frontiers in Computational Neuroscience15, 7.

    Google Scholar

    Herculano-Houzel , S. (2012). The remarkable, yet not extraordinary, human brain as a scaled-up primate brain and its associated cost. Proceedings of the National Academy of Sciences 109(Supplement 1), 10661–10668.

    Google Scholar

    Hiller , J. & Lipson , H. (2012). Automatic design and manufacture of soft robots. IEEE Transactions on Robotics 28(2), 457–466.

    Google Scholar

    Hoefler , T., Alistarh , D., Ben-Nun , T., Dryden , N. & Peste , A. (2021). Sparsity in deep learning: pruning and growth for efficient inference and training in neural networks. arXiv preprint arXiv:2102.00554.

    Google Scholar

    Horibe , K., Walker , K. & Risi , S. (2021). Regenerating soft robots through neural cellular automata. In EuroGP, 36–50.

    Google Scholar

    Iglesias , J., Eriksson , J., Grize , F., Tomassini , M. & Villa , A. E. (2005). Dynamics of pruning in simulated large-scale spiking neural networks. Biosystems 79(1–3), 11–20.

    Google Scholar

    Johnson , M. H. (2001). Functional brain development in humans. Nature Reviews Neuroscience 2(7), 475–483.

    Google Scholar

    Kassahun , Y. & Sommer , G. (2005). Efficient reinforcement learning through evolutionary acquisition of neural topologies. In ESANN, 259–266.

    Google Scholar

    Kriegman , S., Cheney , N. & Bongard , J. (2018). How morphological development can guide evolution. Scientific Reports 8(1), 13934.

    Google Scholar

    Kriegman , S., Cheney , N., Corucci , F. & Bongard , J. C. (2018). Interoceptive robustness through environment-mediated morphological development. arXiv preprint arXiv:1804.02257.

    Google Scholar

    Laughlin , S. B., van Steveninck , R. R. d. R. & Anderson , J. C. (1998). The metabolic cost of neural information. Nature Neuroscience 1(1), 36–41.

    Google Scholar

    Laurenti , L., Patane , A., Wicker , M., Bortolussi , L., Cardelli , L. & Kwiatkowska , M. (2019). Global adversarial robustness guarantees for neural networks.

    Google Scholar

    LeCun , Y., Denker , J. S., Solla , S. A., Howard , R. E. & Jackel , L. D. (1989). Optimal brain damage. In NIPs, 2. Citeseer, 598–605.

    Google Scholar

    Liao , X., Vasilakos , A. V. & He , Y. (2017). Small-world human brain networks: perspectives and challenges. Neuroscience & Biobehavioral Reviews 77, 286–300.

    Google Scholar

    Lin , T., Stich , S. U., Barba , L., Dmitriev , D. & Jaggi , M. (2020). Dynamic model pruning with feedback. In International Conference on Learning Representations. https://openreview.net/forum?id=SJem8lSFwB.

    Google Scholar

    Lipson , H., Sunspiral , V., Bongard , J. & Cheney , N. (2016). On the difficulty of co-optimizing morphology and control in evolved virtual creatures. In Artificial Life Conference Proceedings 13. MIT Press, 226–233.

    Google Scholar

    Liu , S., Chen , T., Chen , X., Atashgahi , Z., Yin , L., Kou , H., Shen , L., Pechenizkiy , M., Wang , Z. & Mocanu , D. C. (2021). Sparse training via boosting pruning plasticity with neuroregeneration. arXiv preprint arXiv:2106.10404.

    Google Scholar

    Liu , Z., Sun , M., Zhou , T., Huang , G. & Darrell , T. (2019). Rethinking the value of network pruning. In International Conference on Learning Representations. https://openreview.net/forum?id=rJlnB3C5Ym.

    Google Scholar

    Low , L. K. & Cheng , H.-J. (2006). Axon pruning: an essential step underlying the developmental plasticity of neuronal connections. Philosophical Transactions of the Royal Society B: Biological Sciences 361(1473), 1531–1544.

    Google Scholar

    Medvet , E., Bartoli , A., De Lorenzo , A. & Fidel , G. (2020). Evolution of distributed neural controllers for voxel-based soft robots. In Proceedings of the 2020 Genetic and Evolutionary Computation Conference, 112–120.

    Google Scholar

    Medvet , E., Bartoli , A., De Lorenzo , A. & Seriani , S. (2020). 2D-VSR-Sim: a simulation tool for the optimization of 2-D voxel-based soft robots. SoftwareX 12.

    Google Scholar

    Medvet , E., Bartoli , A., Pigozzi , F. & Rochelli , M. (2021). Biodiversity in evolved voxel-based soft robots. In Proceedings of the Genetic and Evolutionary Computation Conference, 129–137.

    Google Scholar

    Meunier , D., Lambiotte , R. & Bullmore , E. T. (2010). Modular and hierarchically modular organization of brain networks. Frontiers in Neuroscience 4, 200.

    Google Scholar

    Mordvintsev , A., Randazzo , E., Niklasson , E. & Levin , M. (2020). Growing neural cellular automata. Distill 5(2), e23.

    Google Scholar

    Nadizar , G., Medvet , E., Pellegrino , F. A., Zullich , M. & Nichele , S. (2021). On the effects of pruning on evolved neural controllers for soft robots. In Proceedings of the Genetic and Evolutionary Computation Conference Companion, 1744–1752.

    Google Scholar

    Naumov , M., Chien , L., Vandermersch , P. & Kapasi , U. (2010). Cusparse library. In GPU Technology Conference.

    Google Scholar

    Neyshabur , B., Li , Z., Bhojanapalli , S., LeCun , Y. & Srebro , N. (2019). The role of over-parametrization in generalization of neural networks. In International Conference on Learning Representations. https://openreview.net/forum?id=BygfghAcYX.

    Google Scholar

    Nichele , S., Ose , M. B., Risi , S. & Tufte , G. (2017). Ca-neat: evolved compositional pattern producing networks for cellular automata morphogenesis and replication. IEEE Transactions on Cognitive and Developmental Systems 10(3), 687–700.

    Google Scholar

    Niklasson , E., Mordvintsev , A., Randazzo , E. & Levin , M. (2021). Self-organising textures. Distill 6(2), e00027–003.

    Google Scholar

    Pfeifer , R. & Bongard , J. (2006). How the Body Shapes the Way We Think: A New View of Intelligence. MIT Press.

    Google Scholar

    Pontes-Filho , S. & Nichele , S. (2019). Towards a framework for the evolution of artificial general intelligence. arXiv preprint arXiv:1903.10410.

    Google Scholar

    Power , J. D. & Schlaggar , B. L. (2017). Neural plasticity across the lifespan. Wiley Interdisciplinary Reviews: Developmental Biology 6(1), e216.

    Google Scholar

    Prakash , A., Storer , J., Florencio , D. & Zhang , C. (2019). Repr: Improved training of convolutional filters. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 10666–10675.

    Google Scholar

    Qiu , H., Garratt , M., Howard , D. & Anavatti , S. (2018). Evolving spiking neural networks for nonlinear control problems. In 2018 IEEE Symposium Series on Computational Intelligence (SSCI), 1367–1373, IEEE.

    Google Scholar

    Raman , D. V., Rotondo , A. P. & O’Leary , T. (2019). Fundamental bounds on learning performance in neural circuits. Proceedings of the National Academy of Sciences 116(21), 10537–10546.

    Google Scholar

    Ramesh , A., Pavlov , M., Goh , G., Gray , S., Voss , C., Radford , A., Chen , M. & Sutskever , I. (2021). Zero-shot text-to-image generation. arXiv preprint arXiv:2102.12092.

    Google Scholar

    Real , E., Moore , S., Selle , A., Saxena , S., Suematsu , Y. L., Tan , J., Le , Q. V. & Kurakin , A. (2017). Large-scale evolution of image classifiers. In International Conference on Machine Learning, 2902–2911. PMLR.

    Google Scholar

    Renda , A., Frankle , J. & Carbin , M. (2020). Comparing fine-tuning and rewinding in neural network pruning. In International Conference on Learning Representations.

    Google Scholar

    Riccomagno , M. M. & Kolodkin , A. L. (2015). Sculpting neural circuits by axon and dendrite pruning. Annual Review of Cell and Developmental Biology 31, 779–805.

    Google Scholar

    Sakai , J. (2020). Core concept: how synaptic pruning shapes neural wiring during development and, possibly, in disease. Proceedings of the National Academy of Sciences 117(28), 16096–16099.

    Google Scholar

    Santosa , F. & Symes , W. W. (1986). Linear inversion of band-limited reflection seismograms. SIAM Journal on Scientific and Statistical Computing 7(4), 1307–1330.

    Google Scholar

    Schuldiner , O. & Yaron , A. (2015). Mechanisms of developmental neurite pruning. Cellular and Molecular Life Sciences 72(1), 101–119.

    Google Scholar

    Segev , I. & Schneidman , E. (1999). Axons as computing devices: basic insights gained from models. Journal of Physiology-Paris 93(4), 263–270.

    Google Scholar

    Shi , Y., Nguyen , L., Oh , S., Liu , X. & Kuzum , D. (2019). A soft-pruning method applied during training of spiking neural networks for in-memory computing applications. Frontiers in Neuroscience 13, 405.

    Google Scholar

    Siebel , N. T., Botel , J. & Sommer , G. (2009). Efficient neural network pruning during neuro-evolution. In 2009 International Joint Conference on Neural Networks, 2920–2927, IEEE.

    Google Scholar

    Sporns , O. (2013). Structure and function of complex brain networks. Dialogues in Clinical Neuroscience 15(3), 247.

    Google Scholar

    Sporns , O., Chialvo , D. R., Kaiser , M. & Hilgetag , C. C. (2004). Complex networks: small-world and scale-free architectures. Trends in Cognitive Sciences 9(8), 418–425.

    Google Scholar

    Stanley , K. O. & Miikkulainen , R. (2002). Evolving neural networks through augmenting topologies. Evolutionary Computation 10(2), 99–127.

    Google Scholar

    Strubell , E., Ganesh , A. & McCallum , A. (2019). Energy and policy considerations for deep learning in nlp. arXiv preprint arXiv:1906.02243.

    Google Scholar

    Sudhakaran , S., Grbic , D., Li , S., Katona , A., Najarro , E., Glanois , C. & Risi , S. (2021). Growing 3D artefacts and functional machines with neural cellular automata. arXiv preprint arXiv:2103.08737.

    Google Scholar

    Talamini , J., Medvet , E., Bartoli , A. & De Lorenzo , A. (2019). Evolutionary synthesis of sensing controllers for voxel-based soft robots. In Artificial Life Conference Proceedings, MIT Press, 574–581.

    Google Scholar

    Thimm , G. & Fiesler , E. (1995). Evaluating pruning methods. In Proceedings of the International Symposium on Artificial neural networks, 20–25, Citeseer.

    Google Scholar

    Thodberg , H. H. (1991). Improving generalization of neural networks through pruning. International Journal of Neural Systems 1(04), 317–326.

    Google Scholar

    Tibshirani , R. (1997). The lasso method for variable selection in the cox model. Statistics in medicine 16(4), 385–395.

    Google Scholar

    Variengien , A., Nichele , S., Glover , T. & Pontes-Filho , S. (2021). Towards self-organized control: Using neural cellular automata to robustly control a cart-pole agent. arXiv preprint arXiv:2106.15240.

    Google Scholar

    Vézquez-Rodrguez , B., Liu , Z.-Q., Hagmann , P. & Misic , B. (2020). Signal propagation via cortical hierarchies. Network Neuroscience 4(4), 1072–1090.

    Google Scholar

    Ye , S., Xu , K., Liu , S., Cheng , H., Lambrechts , J.-H., Zhang , H., Zhou , A., Ma , K., Wang , Y. & Lin , X. (2019). Adversarial robustness vs. model compression, or both?. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 111–120.

    Google Scholar

    You , H., Li , C., Xu , P., Fu , Y., Wang , Y., Chen , X., Baraniuk , R. G., Wang , Z. & Lin , Y. (2019). Drawing early-bird tickets: Towards more efficient training of deep networks. arXiv preprint arXiv:1909.11957.

    Google Scholar

    Yuste , R. (2015). From the neuron doctrine to neural networks. Nature Reviews Neuroscience 16(8), 487–497.

    Google Scholar

    Zhang , B.-T. & Mühlenbein , H. (1993). Genetic programming of minimal neural nets using occam’s razor. In Proceedings of the 5th International Conference on Genetic Algorithms (ICGA’93). Citeseer.

    Google Scholar

    Zullich , M., Medvet , E., Pellegrino , F. A. & Ansuini , A. (2021). Speeding-up pruning for artificial neural networks: introducing accelerated iterative magnitude pruning. In 2020 25th International Conference on Pattern Recognition (ICPR), 3868–3875. IEEE.

    Google Scholar

  • Cite this article

    Giorgia Nadizar, Eric Medvet, Hola Huse Ramstad, Stefano Nichele, Felice Andrea Pellegrino, Marco Zullich. 2022. Merging pruning and neuroevolution: towards robust and efficient controllers for modular soft robots. The Knowledge Engineering Review 37(1), doi: 10.1017/S0269888921000151
    Giorgia Nadizar, Eric Medvet, Hola Huse Ramstad, Stefano Nichele, Felice Andrea Pellegrino, Marco Zullich. 2022. Merging pruning and neuroevolution: towards robust and efficient controllers for modular soft robots. The Knowledge Engineering Review 37(1), doi: 10.1017/S0269888921000151

Article Metrics

Article views(78) PDF downloads(42)

RESEARCH ARTICLE   Open Access    

Merging pruning and neuroevolution: towards robust and efficient controllers for modular soft robots

Abstract: Abstract: Artificial neural networks (ANNs) can be employed as controllers for robotic agents. Their structure is often complex, with many neurons and connections, especially when the robots have many sensors and actuators distributed across their bodies and/or when high expressive power is desirable. Pruning (removing neurons or connections) reduces the complexity of the ANN, thus increasing its energy efficiency, and has been reported to improve the generalization capability, in some cases. In addition, it is well-known that pruning in biological neural networks plays a fundamental role in the development of brains and their ability to learn. In this study, we consider the evolutionary optimization of neural controllers for the case study of Voxel-based soft robots, a kind of modular, bio-inspired soft robots, applying pruning during fitness evaluation. For a locomotion task, and for centralized as well as distributed controllers, we experimentally characterize the effect of different forms of pruning on after-pruning effectiveness, life-long effectiveness, adaptability to new terrains, and behavior. We find that incorporating some forms of pruning in neuroevolution leads to almost equally effective controllers as those evolved without pruning, with the benefit of higher robustness to pruning. We also observe occasional improvements in generalization ability.

    • The experimental evaluation of this work has been supported by a Google Faculty Research Award granted to E.M. and has been partially done on CINECA HPC cluster within the CINECA-University of Trieste agreement. This work was partially funded by the Norwegian Research Council (NFR) through their IKTPLUSS research and innovation action under the project Socrates (grant agreement 270961) and Young Research Talent program under the project DeepCA (grant agreement 286558). G.N. was supported by NordSTAR - Nordic Center for Sustainable and Trustworthy AI Research (OsloMet Project Code 202237-100).

    • The author(s) declare none.

    • The online version of this article has been updated since its original publication. A notice detailing the changes has been published at: https://doi.org/10.1017/S0269888922000017

    • © The Author(s), 2022. Published by Cambridge University Press2022Cambridge University Press
References (86)
  • About this article
    Cite this article
    Giorgia Nadizar, Eric Medvet, Hola Huse Ramstad, Stefano Nichele, Felice Andrea Pellegrino, Marco Zullich. 2022. Merging pruning and neuroevolution: towards robust and efficient controllers for modular soft robots. The Knowledge Engineering Review 37(1), doi: 10.1017/S0269888921000151
    Giorgia Nadizar, Eric Medvet, Hola Huse Ramstad, Stefano Nichele, Felice Andrea Pellegrino, Marco Zullich. 2022. Merging pruning and neuroevolution: towards robust and efficient controllers for modular soft robots. The Knowledge Engineering Review 37(1), doi: 10.1017/S0269888921000151
  • Catalog

      /

      DownLoad:  Full-Size Img  PowerPoint
      Return
      Return