In an article published today in the journal Scientific Reports, researchers show how these two challenges are solved by adopting a physical concept that was introduced a century ago to describe the formation of a magnet during a process of iron bulk cooling.

Using a careful optimization procedure and exhaustive simulations, a group of scientists from Bar-Ilan University has demonstrated the usefulness of the physical concept of power-law scaling to deep learning. This central concept in physics, which arises from diverse phenomena, including the timing and magnitude of earthquakes, Internet topology and social networks, stock price fluctuations, word frequencies in linguistics, and signal amplitudes in brain activity, has also been found to be applicable in the ever-growing field of AI, and especially deep learning.

"Test errors with online learning, where each example is trained only once, are in close agreement with state-of-the-art algorithms consisting of a very large number of epochs, where each example is trained many times. This result has an important implication on rapid decision making such as robotic control," said Prof. Ido Kanter, of Bar-Ilan's Department of Physics and Gonda (Goldshmied) Multidisciplinary Brain Research Center, who led the research. "The power-law scaling, governing different dynamical rules and network architectures, enables the classification and hierarchy creation among the different examined classification or decision problems," he added.

"One of the important ingredients of the advanced deep learning algorithm is the recent new bridge between experimental neuroscience and advanced artificial intelligence learning algorithms," said PhD student Shira Sardi, a co-author of the study. Our new type of experiments on neuronal cultures indicate that an increase in the training frequency enables us to significantly accelerate the neuronal adaptation process. "This accelerated brain-inspired mechanism enables building advanced deep learning algorithms which outperform existing ones," said PhD student Yuval Meir, another co-author.

The reconstructed bridge from physics and experimental neuroscience to machine learning is expected to advance artificial intelligence and especially ultrafast decision making under limited training examples as to contribute to the formation of a theoretical framework of the field of deep learning.

Story Source:

Materials provided by Bar-Ilan University. Note: Content may be edited for style and length.

Journal Reference:

1. Yuval Meir, Shira Sardi, Shiri Hodassman, Karin Kisos, Itamar Ben-Noam, Amir Goldental, Ido Kanter. Power-law scaling to assist with key challenges in artificial intelligence. Scientific Reports, 2020; 10 (1) DOI: 10.1038/s41598-020-76764-1